Sep 19 05:38:59.273344 ip-10-0-142-57 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 19 05:38:59.273353 ip-10-0-142-57 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 19 05:38:59.273360 ip-10-0-142-57 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 19 05:38:59.273573 ip-10-0-142-57 systemd[1]: Failed to start Kubernetes Kubelet. Sep 19 05:39:09.522524 ip-10-0-142-57 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 19 05:39:09.522540 ip-10-0-142-57 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 1a9e403e46804200930e3e154d4c7541 -- Sep 19 05:41:10.430816 ip-10-0-142-57 systemd[1]: Starting Kubernetes Kubelet... Sep 19 05:41:14.038897 ip-10-0-142-57 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 19 05:41:14.038897 ip-10-0-142-57 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 19 05:41:14.038897 ip-10-0-142-57 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 19 05:41:14.038897 ip-10-0-142-57 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 19 05:41:14.038897 ip-10-0-142-57 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 19 05:41:14.039845 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.038982 2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 19 05:41:14.042000 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.041980 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 05:41:14.042000 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.041999 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042003 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042010 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042015 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042018 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042022 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042025 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042033 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042037 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042040 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042043 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042046 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042048 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042052 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042054 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042058 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042061 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042063 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042066 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 05:41:14.042066 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042069 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042072 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042075 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042079 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042082 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042085 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042088 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042090 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042093 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042096 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042098 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042101 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042104 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042107 2584 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042110 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042113 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042115 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042118 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042121 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042123 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 05:41:14.042601 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042126 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042129 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042132 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042136 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042139 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042143 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042146 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042149 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042152 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042155 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042158 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042161 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042164 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042167 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042171 2584 feature_gate.go:328] unrecognized feature gate: Example Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042175 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042178 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042183 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042187 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 05:41:14.043123 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042190 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042193 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042196 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042198 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042201 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042204 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042206 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042209 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042212 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042215 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042218 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042221 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042223 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042226 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042228 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042232 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042235 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042238 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042241 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042244 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 05:41:14.043638 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042247 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042250 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042254 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042257 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042259 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042262 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042265 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042269 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042272 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042275 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042278 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042281 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042283 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042286 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042289 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042292 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042294 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042297 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042299 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 05:41:14.044171 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.042302 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043082 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043090 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043093 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043099 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043103 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043107 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043110 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043114 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043117 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043121 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043124 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043128 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043132 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043135 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043139 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043142 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043145 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043148 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 05:41:14.044679 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043152 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043155 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043158 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043161 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043164 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043167 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043170 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043173 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043176 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043178 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043181 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043184 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043187 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043189 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043192 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043195 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043198 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043201 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043205 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043208 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 05:41:14.045180 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043211 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043213 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043217 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043250 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043255 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043259 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043262 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043265 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043268 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043272 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043276 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043279 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043282 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043286 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043289 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043292 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043295 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043298 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043301 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 05:41:14.045715 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043304 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043307 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043309 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043314 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043317 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043320 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043323 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043325 2584 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043328 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043332 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043335 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043337 2584 feature_gate.go:328] unrecognized feature gate: Example Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043340 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043343 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043346 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043349 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043353 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043356 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043359 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043362 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 05:41:14.046240 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043365 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043367 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043370 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043372 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043375 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043378 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043388 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043393 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043396 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043399 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043402 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043408 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043411 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043414 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043417 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043420 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043422 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043425 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043428 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043431 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 05:41:14.046792 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043434 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043436 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043911 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043917 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043921 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043925 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043929 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043932 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043935 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043939 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043942 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043945 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043949 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043953 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043957 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043961 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043964 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043968 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043984 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 05:41:14.047324 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043988 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043990 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043993 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043996 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.043999 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044002 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044005 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044008 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044012 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044015 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044018 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044022 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044026 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044030 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044033 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044037 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044041 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044043 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044046 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 05:41:14.047874 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044049 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044052 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044055 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044058 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044061 2584 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044064 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044067 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044071 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044073 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044076 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044079 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044082 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044084 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044087 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044090 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044093 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044095 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044098 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044101 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044104 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 05:41:14.048380 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044106 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044109 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044112 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044114 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044117 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044120 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044122 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044128 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044131 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044135 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044138 2584 feature_gate.go:328] unrecognized feature gate: Example Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044141 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044144 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044147 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044150 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044153 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044157 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044161 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044166 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044169 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 05:41:14.048929 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044172 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044175 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044179 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044182 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044185 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044188 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044191 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044194 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044196 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044199 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044202 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044205 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044207 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044210 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044213 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044216 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044219 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044222 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044224 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044228 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 05:41:14.049513 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044231 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044233 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.044237 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044305 2584 flags.go:64] FLAG: --address="0.0.0.0" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044346 2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044353 2584 flags.go:64] FLAG: --anonymous-auth="true" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044357 2584 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044362 2584 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044365 2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044370 2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044376 2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044380 2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044383 2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044388 2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044391 2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044395 2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044398 2584 flags.go:64] FLAG: --cgroup-root="" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044401 2584 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044404 2584 flags.go:64] FLAG: --client-ca-file="" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044408 2584 flags.go:64] FLAG: --cloud-provider="external" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044411 2584 flags.go:64] FLAG: --cluster-dns="[]" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044417 2584 flags.go:64] FLAG: --cluster-domain="" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044420 2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044423 2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 19 05:41:14.050049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044427 2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044430 2584 flags.go:64] FLAG: --container-log-max-files="5" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044434 2584 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044438 2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044441 2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044445 2584 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044448 2584 flags.go:64] FLAG: --contention-profiling="false" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044452 2584 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044455 2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044459 2584 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044463 2584 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044468 2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044486 2584 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044490 2584 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044493 2584 flags.go:64] FLAG: --enable-load-reader="false" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044497 2584 flags.go:64] FLAG: --enable-server="true" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044500 2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044505 2584 flags.go:64] FLAG: --event-burst="100" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044511 2584 flags.go:64] FLAG: --event-qps="50" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044514 2584 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044518 2584 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044521 2584 flags.go:64] FLAG: --eviction-hard="" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044525 2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044528 2584 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044532 2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 19 05:41:14.050687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044535 2584 flags.go:64] FLAG: --eviction-soft="" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044538 2584 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044542 2584 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044545 2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044548 2584 flags.go:64] FLAG: --experimental-mounter-path="" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044552 2584 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044555 2584 flags.go:64] FLAG: --fail-swap-on="true" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044558 2584 flags.go:64] FLAG: --feature-gates="" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044563 2584 flags.go:64] FLAG: --file-check-frequency="20s" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044566 2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044569 2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044573 2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044577 2584 flags.go:64] FLAG: --healthz-port="10248" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044581 2584 flags.go:64] FLAG: --help="false" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044585 2584 flags.go:64] FLAG: --hostname-override="ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044588 2584 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044592 2584 flags.go:64] FLAG: --http-check-frequency="20s" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044595 2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044598 2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044603 2584 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044606 2584 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044609 2584 flags.go:64] FLAG: --image-service-endpoint="" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044612 2584 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 19 05:41:14.051329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044616 2584 flags.go:64] FLAG: --kube-api-burst="100" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044619 2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044624 2584 flags.go:64] FLAG: --kube-api-qps="50" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044627 2584 flags.go:64] FLAG: --kube-reserved="" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044630 2584 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044633 2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044637 2584 flags.go:64] FLAG: --kubelet-cgroups="" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044640 2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044644 2584 flags.go:64] FLAG: --lock-file="" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044647 2584 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044650 2584 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044653 2584 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044659 2584 flags.go:64] FLAG: --log-json-split-stream="false" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044662 2584 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044665 2584 flags.go:64] FLAG: --log-text-split-stream="false" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044668 2584 flags.go:64] FLAG: --logging-format="text" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044671 2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044675 2584 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044678 2584 flags.go:64] FLAG: --manifest-url="" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044681 2584 flags.go:64] FLAG: --manifest-url-header="" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044686 2584 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044690 2584 flags.go:64] FLAG: --max-open-files="1000000" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044694 2584 flags.go:64] FLAG: --max-pods="110" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044698 2584 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044701 2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 19 05:41:14.052073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044704 2584 flags.go:64] FLAG: --memory-manager-policy="None" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044708 2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044711 2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044714 2584 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044719 2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044727 2584 flags.go:64] FLAG: --node-status-max-images="50" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044730 2584 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044734 2584 flags.go:64] FLAG: --oom-score-adj="-999" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044737 2584 flags.go:64] FLAG: --pod-cidr="" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044742 2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044748 2584 flags.go:64] FLAG: --pod-manifest-path="" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044751 2584 flags.go:64] FLAG: --pod-max-pids="-1" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044754 2584 flags.go:64] FLAG: --pods-per-core="0" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044758 2584 flags.go:64] FLAG: --port="10250" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044761 2584 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044764 2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-011dcf895d6f788d0" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044768 2584 flags.go:64] FLAG: --qos-reserved="" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044771 2584 flags.go:64] FLAG: --read-only-port="10255" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044774 2584 flags.go:64] FLAG: --register-node="true" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044777 2584 flags.go:64] FLAG: --register-with-taints="" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044782 2584 flags.go:64] FLAG: --registry-burst="10" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044785 2584 flags.go:64] FLAG: --registry-qps="5" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044788 2584 flags.go:64] FLAG: --reserved-cpus="" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044791 2584 flags.go:64] FLAG: --reserved-memory="" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044795 2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 19 05:41:14.052788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044798 2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044802 2584 flags.go:64] FLAG: --rotate-certificates="false" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044805 2584 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044808 2584 flags.go:64] FLAG: --runonce="false" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044811 2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044816 2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044819 2584 flags.go:64] FLAG: --seccomp-default="false" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044822 2584 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044825 2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044829 2584 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044832 2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044836 2584 flags.go:64] FLAG: --storage-driver-password="root" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044840 2584 flags.go:64] FLAG: --storage-driver-secure="false" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044843 2584 flags.go:64] FLAG: --storage-driver-table="stats" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044846 2584 flags.go:64] FLAG: --storage-driver-user="root" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044850 2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044856 2584 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044860 2584 flags.go:64] FLAG: --system-cgroups="" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044863 2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044869 2584 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044873 2584 flags.go:64] FLAG: --tls-cert-file="" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044876 2584 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044881 2584 flags.go:64] FLAG: --tls-min-version="" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044884 2584 flags.go:64] FLAG: --tls-private-key-file="" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044887 2584 flags.go:64] FLAG: --topology-manager-policy="none" Sep 19 05:41:14.053549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044890 2584 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044893 2584 flags.go:64] FLAG: --topology-manager-scope="container" Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044897 2584 flags.go:64] FLAG: --v="2" Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044923 2584 flags.go:64] FLAG: --version="false" Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044928 2584 flags.go:64] FLAG: --vmodule="" Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044932 2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.044936 2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045060 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045064 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045067 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045072 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045076 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045080 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045083 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045086 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045089 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045092 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045094 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045097 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045100 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045104 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 05:41:14.054646 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045107 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045110 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045114 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045117 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045120 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045123 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045126 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045129 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045132 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045134 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045137 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045140 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045143 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045146 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045149 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045152 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045156 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045160 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045162 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 05:41:14.055209 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045165 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045168 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045171 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045174 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045179 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045182 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045185 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045188 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045191 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045194 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045197 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045200 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045203 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045206 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045210 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045216 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045219 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045222 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045226 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 05:41:14.055718 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045229 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045231 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045234 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045237 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045240 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045243 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045246 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045248 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045251 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045254 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045257 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045260 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045262 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045265 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045268 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045271 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045274 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045278 2584 feature_gate.go:328] unrecognized feature gate: Example Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045281 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045284 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 05:41:14.056230 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045287 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045290 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045293 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045296 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045299 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045303 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045306 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045309 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045316 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045319 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045322 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045325 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045328 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045331 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045333 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045336 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045339 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045342 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045345 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 05:41:14.056795 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045347 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045350 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045367 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045370 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045374 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045378 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045381 2584 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.045384 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.045837 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.054411 2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.054434 2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.054464 2584 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.054492 2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054854 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054867 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 05:41:14.057300 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054871 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054874 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054877 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054880 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054884 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054887 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054891 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054897 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054900 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054904 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054907 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054910 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054912 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054915 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054919 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054922 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054925 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054928 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054930 2584 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 05:41:14.057772 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054934 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054936 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054939 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054942 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054945 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054948 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054952 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054955 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054958 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054961 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054964 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054967 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054970 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054974 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054976 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054979 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054982 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054985 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054988 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054991 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 05:41:14.058318 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054993 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054996 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.054999 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055003 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055007 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055010 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055013 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055017 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055020 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055022 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055025 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055028 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055031 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055034 2584 feature_gate.go:328] unrecognized feature gate: Example Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055037 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055040 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055043 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055046 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055049 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055051 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 05:41:14.058946 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055054 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055058 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055062 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055065 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055070 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055073 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055076 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055078 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055081 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055084 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055086 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055089 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055092 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055095 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055097 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055100 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055103 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055105 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055110 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055112 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 05:41:14.059492 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055115 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055118 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055121 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055123 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055126 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055129 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055132 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055135 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055138 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055140 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055143 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055146 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055149 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055152 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055155 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055157 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055161 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 05:41:14.060021 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055164 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.055170 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055301 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055307 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055311 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055314 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055318 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055321 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055324 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055327 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055330 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055333 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055336 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055338 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 05:41:14.060471 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055342 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055345 2584 feature_gate.go:328] unrecognized feature gate: Example Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055348 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055351 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055354 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055358 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055361 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055365 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055368 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055370 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055373 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055376 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055379 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055382 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055385 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055389 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055392 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055396 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055400 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 05:41:14.060862 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055403 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055406 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055409 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055412 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055415 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055418 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055421 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055423 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055426 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055429 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055432 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055434 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055437 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055440 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055443 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055446 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055448 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055451 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055454 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055456 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 05:41:14.061370 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055459 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055462 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055465 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055468 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055470 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055493 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055497 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055499 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055502 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055505 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055508 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055511 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055514 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055517 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055519 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055522 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055525 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055528 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055531 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055534 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 05:41:14.061912 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055537 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055540 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055542 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055545 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055548 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055551 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055554 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055557 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055560 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055563 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055566 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055569 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055572 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055575 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055578 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055581 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055584 2584 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055586 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055589 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055592 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 05:41:14.062443 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055595 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055598 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055601 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055604 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055607 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055609 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055612 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:14.055615 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.055621 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.055755 2584 server.go:962] "Client rotation is on, will bootstrap in background" Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.057818 2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.058927 2584 server.go:1019] "Starting client certificate rotation" Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.059034 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 19 05:41:14.063039 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.059754 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 19 05:41:14.063396 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.061524 2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 19 05:41:14.063396 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.062786 2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 19 05:41:14.068566 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.068546 2584 log.go:25] "Validated CRI v1 runtime API" Sep 19 05:41:14.070254 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.070239 2584 log.go:25] "Validated CRI v1 image API" Sep 19 05:41:14.071236 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.071216 2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 19 05:41:14.073752 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.073726 2584 fs.go:135] Filesystem UUIDs: map[576b00e8-305e-43f3-800a-898d9bf078cf:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f1ddc29c-55c7-4d39-8737-74d643b2159e:/dev/nvme0n1p4] Sep 19 05:41:14.073829 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.073751 2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 19 05:41:14.079956 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.079824 2584 manager.go:216] Machine: {Timestamp:2026-09-19 05:41:14.079282612 +0000 UTC m=+2.072510138 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099454 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec271a5e479b31073d7f4a7d02a163bd SystemUUID:ec271a5e-479b-3107-3d7f-4a7d02a163bd BootID:1a9e403e-4680-4200-930e-3e154d4c7541 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c4:81:30:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c4:81:30:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:d8:8d:7a:69:5e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 19 05:41:14.079956 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.079948 2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 19 05:41:14.080077 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.080036 2584 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 19 05:41:14.081601 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.081572 2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 19 05:41:14.081760 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.081604 2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-57.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 19 05:41:14.082421 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.082410 2584 topology_manager.go:138] "Creating topology manager with none policy" Sep 19 05:41:14.082455 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.082424 2584 container_manager_linux.go:309] "Creating device plugin manager" Sep 19 05:41:14.082455 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.082434 2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 19 05:41:14.082455 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.082447 2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 19 05:41:14.083595 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.083583 2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 19 05:41:14.087545 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.087531 2584 state_mem.go:36] "Initialized new in-memory state store" Sep 19 05:41:14.087699 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.087689 2584 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 19 05:41:14.087747 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.087737 2584 kubelet.go:486] "Attempting to sync node with API server" Sep 19 05:41:14.087778 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.087757 2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 19 05:41:14.087778 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.087775 2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 19 05:41:14.087837 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.087786 2584 kubelet.go:398] "Adding apiserver pod source" Sep 19 05:41:14.087868 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.087846 2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 19 05:41:14.088407 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.088392 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 19 05:41:14.088494 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.088414 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 19 05:41:14.093036 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.093012 2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 19 05:41:14.097754 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.097736 2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 19 05:41:14.097828 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.097770 2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 19 05:41:14.098802 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098786 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 19 05:41:14.098802 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098805 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 19 05:41:14.098930 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098814 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 19 05:41:14.098930 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098822 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 19 05:41:14.098930 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098835 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 19 05:41:14.098930 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098844 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 19 05:41:14.098930 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098872 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 19 05:41:14.098930 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098880 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 19 05:41:14.098930 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098888 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 19 05:41:14.098930 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098894 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 19 05:41:14.098930 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098905 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 19 05:41:14.098930 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098912 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 19 05:41:14.099225 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098943 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 19 05:41:14.099225 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.098948 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 19 05:41:14.102954 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.102938 2584 server.go:1268] "Started kubelet" Sep 19 05:41:14.103091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.103066 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 19 05:41:14.103212 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.103095 2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 19 05:41:14.103271 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.103021 2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 19 05:41:14.103327 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.103289 2584 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 19 05:41:14.103431 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.103398 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 19 05:41:14.103566 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.103533 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-57.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 19 05:41:14.103670 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.103620 2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-57.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 19 05:41:14.103987 ip-10-0-142-57 systemd[1]: Started Kubernetes Kubelet. Sep 19 05:41:14.104934 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.104919 2584 server.go:310] "Adding debug handlers to kubelet server" Sep 19 05:41:14.107359 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.107341 2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 19 05:41:14.107922 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.107901 2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 19 05:41:14.112948 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.112932 2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 19 05:41:14.112948 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.112936 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 19 05:41:14.113292 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.113123 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:14.113677 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.113656 2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 19 05:41:14.113841 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.113822 2584 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 19 05:41:14.113913 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.113845 2584 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 19 05:41:14.113985 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.113969 2584 factory.go:55] Registering systemd factory Sep 19 05:41:14.114047 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.113991 2584 factory.go:223] Registration of the systemd container factory successfully Sep 19 05:41:14.114047 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.113995 2584 reconstruct.go:97] "Volume reconstruction finished" Sep 19 05:41:14.114047 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.114006 2584 reconciler.go:29] "Reconciler: start to sync state" Sep 19 05:41:14.114224 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.114209 2584 factory.go:153] Registering CRI-O factory Sep 19 05:41:14.114274 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.114229 2584 factory.go:223] Registration of the crio container factory successfully Sep 19 05:41:14.114309 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.114301 2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 19 05:41:14.114366 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.114324 2584 factory.go:103] Registering Raw factory Sep 19 05:41:14.114366 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.114341 2584 manager.go:1196] Started watching for new ooms in manager Sep 19 05:41:14.114832 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.114817 2584 manager.go:349] Starting recovery of all containers Sep 19 05:41:14.118743 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.118709 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 19 05:41:14.118872 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.118833 2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-57.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 19 05:41:14.124174 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.124153 2584 manager.go:354] Recovery completed Sep 19 05:41:14.136681 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.136642 2584 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Sep 19 05:41:14.140970 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.140951 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 05:41:14.144916 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.144892 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 05:41:14.145025 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.144935 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 05:41:14.145025 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.144951 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 05:41:14.145556 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.145536 2584 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 19 05:41:14.145556 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.145551 2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 19 05:41:14.145695 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.145572 2584 state_mem.go:36] "Initialized new in-memory state store" Sep 19 05:41:14.145695 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.145582 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 05:41:14.146442 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.146422 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 05:41:14.146531 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.146452 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 05:41:14.146531 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.146466 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 05:41:14.148008 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.147990 2584 policy_none.go:49] "None policy: Start" Sep 19 05:41:14.148091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.148011 2584 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 19 05:41:14.148091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.148026 2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 19 05:41:14.151813 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.151798 2584 policy_none.go:47] "Start" Sep 19 05:41:14.156257 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.156229 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-frgbp" Sep 19 05:41:14.159517 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.158385 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-57.ec2.internal.18d6a269132690f3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-57.ec2.internal,UID:ip-10-0-142-57.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-57.ec2.internal,},FirstTimestamp:2026-09-19 05:41:14.102911219 +0000 UTC m=+2.096138744,LastTimestamp:2026-09-19 05:41:14.102911219 +0000 UTC m=+2.096138744,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-57.ec2.internal,}" Sep 19 05:41:14.173241 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.173012 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-57.ec2.internal.18d6a26915a782f3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-57.ec2.internal,UID:ip-10-0-142-57.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-57.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-57.ec2.internal,},FirstTimestamp:2026-09-19 05:41:14.144916211 +0000 UTC m=+2.138143744,LastTimestamp:2026-09-19 05:41:14.144916211 +0000 UTC m=+2.138143744,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-57.ec2.internal,}" Sep 19 05:41:14.179165 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.179145 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-frgbp" Sep 19 05:41:14.186173 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.186149 2584 manager.go:337] "Starting Device Plugin manager" Sep 19 05:41:14.207623 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.186244 2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 19 05:41:14.207623 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.186261 2584 server.go:87] "Starting device plugin registration server" Sep 19 05:41:14.207623 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.186579 2584 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 19 05:41:14.207623 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.186605 2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 19 05:41:14.207623 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.186692 2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 19 05:41:14.207623 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.186777 2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 19 05:41:14.207623 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.186784 2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 19 05:41:14.207623 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.187372 2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 19 05:41:14.207623 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.187444 2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:14.258837 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.258796 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 19 05:41:14.260085 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.260056 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 19 05:41:14.260214 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.260134 2584 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 19 05:41:14.260214 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.260172 2584 kubelet.go:2448] "Starting kubelet main sync loop" Sep 19 05:41:14.260385 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.260221 2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 19 05:41:14.262510 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.262490 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 05:41:14.287718 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.287687 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 05:41:14.289443 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.289366 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 05:41:14.289443 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.289426 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 05:41:14.289443 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.289440 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 05:41:14.289709 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.289465 2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.298312 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.298293 2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.298403 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.298319 2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-57.ec2.internal\": node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:14.320754 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.320722 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:14.361111 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.361077 2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-57.ec2.internal"] Sep 19 05:41:14.361210 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.361167 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 05:41:14.362280 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.362264 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 05:41:14.362344 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.362298 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 05:41:14.362344 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.362313 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 05:41:14.364703 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.364690 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 05:41:14.364824 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.364808 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.364870 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.364840 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 05:41:14.365633 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.365613 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 05:41:14.365747 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.365645 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 05:41:14.365747 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.365663 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 05:41:14.365747 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.365613 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 05:41:14.365747 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.365718 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 05:41:14.365747 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.365729 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 05:41:14.367972 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.367958 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.368043 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.367985 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 05:41:14.368773 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.368757 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 05:41:14.368865 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.368784 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 05:41:14.368865 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.368794 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 05:41:14.386194 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.386164 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-57.ec2.internal\" not found" node="ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.390714 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.390694 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-57.ec2.internal\" not found" node="ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.415954 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.415915 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6fb336e699793f1bfd1aad113e3eb3a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal\" (UID: \"a6fb336e699793f1bfd1aad113e3eb3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.415954 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.415954 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6fb336e699793f1bfd1aad113e3eb3a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal\" (UID: \"a6fb336e699793f1bfd1aad113e3eb3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.416158 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.415981 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f782f89f5d636e4550fa0cef2186e194-config\") pod \"kube-apiserver-proxy-ip-10-0-142-57.ec2.internal\" (UID: \"f782f89f5d636e4550fa0cef2186e194\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.420968 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.420948 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:14.517197 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.517164 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6fb336e699793f1bfd1aad113e3eb3a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal\" (UID: \"a6fb336e699793f1bfd1aad113e3eb3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.517300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.517204 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6fb336e699793f1bfd1aad113e3eb3a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal\" (UID: \"a6fb336e699793f1bfd1aad113e3eb3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.517300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.517223 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f782f89f5d636e4550fa0cef2186e194-config\") pod \"kube-apiserver-proxy-ip-10-0-142-57.ec2.internal\" (UID: \"f782f89f5d636e4550fa0cef2186e194\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.517300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.517277 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6fb336e699793f1bfd1aad113e3eb3a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal\" (UID: \"a6fb336e699793f1bfd1aad113e3eb3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.517300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.517281 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6fb336e699793f1bfd1aad113e3eb3a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal\" (UID: \"a6fb336e699793f1bfd1aad113e3eb3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.517469 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.517322 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f782f89f5d636e4550fa0cef2186e194-config\") pod \"kube-apiserver-proxy-ip-10-0-142-57.ec2.internal\" (UID: \"f782f89f5d636e4550fa0cef2186e194\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.521621 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.521597 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:14.621883 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.621787 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:14.694555 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.694510 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.694681 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:14.694520 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" Sep 19 05:41:14.722592 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.722556 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:14.822719 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.822674 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:14.923287 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:14.923212 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:15.023543 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:15.023512 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:15.058717 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.058680 2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 19 05:41:15.059211 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.058883 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 19 05:41:15.123760 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:15.123727 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:15.185360 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.185267 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 05:36:14 +0000 UTC" deadline="2028-07-02 11:25:33.873178214 +0000 UTC" Sep 19 05:41:15.185360 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.185302 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15653h44m18.687879269s" Sep 19 05:41:15.185360 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.185321 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 05:41:15.186383 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.186364 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 05:41:15.186523 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.186395 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 05:41:15.186523 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.186422 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 05:41:15.224711 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:15.224683 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:15.226108 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.226080 2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 05:41:15.324811 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:15.324764 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-57.ec2.internal\" not found" Sep 19 05:41:15.396282 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.396254 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 05:41:15.413321 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.413287 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" Sep 19 05:41:15.447653 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.447602 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 19 05:41:15.449448 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.449159 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-57.ec2.internal" Sep 19 05:41:15.459357 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.459337 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 19 05:41:15.482464 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:15.482427 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf782f89f5d636e4550fa0cef2186e194.slice/crio-6c884d136f394373337cc5056bbf5d42ae99fa7ecca2a4ce6a6fc561b394a406 WatchSource:0}: Error finding container 6c884d136f394373337cc5056bbf5d42ae99fa7ecca2a4ce6a6fc561b394a406: Status 404 returned error can't find the container with id 6c884d136f394373337cc5056bbf5d42ae99fa7ecca2a4ce6a6fc561b394a406 Sep 19 05:41:15.482838 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:15.482820 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6fb336e699793f1bfd1aad113e3eb3a.slice/crio-0f0f2079a4d06908545c3fbe0440ce4d58cbf0de436cb9450f6110195fa5edd7 WatchSource:0}: Error finding container 0f0f2079a4d06908545c3fbe0440ce4d58cbf0de436cb9450f6110195fa5edd7: Status 404 returned error can't find the container with id 0f0f2079a4d06908545c3fbe0440ce4d58cbf0de436cb9450f6110195fa5edd7 Sep 19 05:41:15.487579 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.487513 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 05:41:15.586006 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.585519 2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 05:41:15.894319 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:15.894229 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 05:41:16.090863 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.090828 2584 apiserver.go:52] "Watching apiserver" Sep 19 05:41:16.096817 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.096197 2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 19 05:41:16.096817 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.096756 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-7jfqn","openshift-image-registry/node-ca-5fm2w","openshift-multus/multus-7lg7k","openshift-multus/multus-additional-cni-plugins-gss8z","openshift-ovn-kubernetes/ovnkube-node-tzjr9","kube-system/kube-apiserver-proxy-ip-10-0-142-57.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm","openshift-cluster-node-tuning-operator/tuned-84zqx","openshift-dns/node-resolver-mmcd6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal","openshift-multus/network-metrics-daemon-8947n","openshift-network-diagnostics/network-check-target-pnp4s","openshift-network-operator/iptables-alerter-xxswm","kube-system/global-pull-secret-syncer-s9kmn"] Sep 19 05:41:16.099837 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.099424 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.101730 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.101708 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xxswm" Sep 19 05:41:16.102644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.102499 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 19 05:41:16.102644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.102510 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wdx4s\"" Sep 19 05:41:16.102644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.102526 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 19 05:41:16.102644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.102517 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 19 05:41:16.104020 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.104000 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lmx88\"" Sep 19 05:41:16.104481 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.104460 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 19 05:41:16.104621 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.104604 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 19 05:41:16.104802 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.104785 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.105204 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.105182 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 19 05:41:16.106968 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.106489 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.107073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.106974 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 19 05:41:16.107073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.106978 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 19 05:41:16.107193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.107132 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 19 05:41:16.107552 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.107420 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 19 05:41:16.107645 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.107392 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7wpl7\"" Sep 19 05:41:16.109274 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.109080 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 19 05:41:16.109351 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.109272 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-db2d6\"" Sep 19 05:41:16.110192 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.109633 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.110192 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.109741 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 19 05:41:16.112518 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.112196 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:16.112518 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.112277 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:16.113133 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.112692 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 19 05:41:16.113133 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.112717 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 19 05:41:16.113133 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.112734 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 19 05:41:16.113133 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.112992 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 19 05:41:16.113365 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.113170 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 19 05:41:16.113365 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.113181 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 19 05:41:16.113365 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.113347 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c95xg\"" Sep 19 05:41:16.114391 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.114373 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 19 05:41:16.116036 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.115613 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mmcd6" Sep 19 05:41:16.117751 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.117579 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 19 05:41:16.117906 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.117857 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-28qr6\"" Sep 19 05:41:16.118384 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.118098 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 19 05:41:16.118384 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.118153 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7jfqn" Sep 19 05:41:16.120876 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.120809 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5fm2w" Sep 19 05:41:16.124161 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.124132 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 19 05:41:16.124859 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.124838 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.124970 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.124930 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sbt5m\"" Sep 19 05:41:16.125030 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125013 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125342 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125544 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qpkfm\"" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125671 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-etc-selinux\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125710 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125720 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xfmz\" (UniqueName: \"kubernetes.io/projected/9355f974-6938-4adf-9c75-280336afb66f-kube-api-access-4xfmz\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125752 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78tgj\" (UniqueName: \"kubernetes.io/projected/11de77e7-d493-4516-be47-51e244b1c3fe-kube-api-access-78tgj\") pod \"node-resolver-mmcd6\" (UID: \"11de77e7-d493-4516-be47-51e244b1c3fe\") " pod="openshift-dns/node-resolver-mmcd6" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125781 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q48hm\" (UniqueName: \"kubernetes.io/projected/a67276d8-00ac-46dd-b864-cdd45f909348-kube-api-access-q48hm\") pod \"iptables-alerter-xxswm\" (UID: \"a67276d8-00ac-46dd-b864-cdd45f909348\") " pod="openshift-network-operator/iptables-alerter-xxswm" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125810 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-run-systemd\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125850 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-run-k8s-cni-cncf-io\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125889 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/653a14cf-5131-4a36-8c13-35f8ab9d499a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125922 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-log-socket\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.125972 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-kubelet\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.126102 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-run-netns\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.126134 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-system-cni-dir\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.126162 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-var-lib-cni-bin\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.126209 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-var-lib-cni-multus\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.126246 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/653a14cf-5131-4a36-8c13-35f8ab9d499a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.126486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.126268 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 19 05:41:16.127392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.126275 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-cni-bin\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.127392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.126329 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11de77e7-d493-4516-be47-51e244b1c3fe-hosts-file\") pod \"node-resolver-mmcd6\" (UID: \"11de77e7-d493-4516-be47-51e244b1c3fe\") " pod="openshift-dns/node-resolver-mmcd6" Sep 19 05:41:16.127392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.126437 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-run-ovn-kubernetes\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.127392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.126944 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:16.127392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.126984 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-var-lib-kubelet\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.127392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127032 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-multus-socket-dir-parent\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.127392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127104 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/653a14cf-5131-4a36-8c13-35f8ab9d499a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.127392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127161 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/795198a1-531e-4263-89f7-59d4a963ef34-ovnkube-config\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.127392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127198 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11de77e7-d493-4516-be47-51e244b1c3fe-tmp-dir\") pod \"node-resolver-mmcd6\" (UID: \"11de77e7-d493-4516-be47-51e244b1c3fe\") " pod="openshift-dns/node-resolver-mmcd6" Sep 19 05:41:16.127392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127271 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-registration-dir\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.127392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127307 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-var-lib-openvswitch\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.127392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127377 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/795198a1-531e-4263-89f7-59d4a963ef34-env-overrides\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.127982 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127459 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-multus-cni-dir\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.127982 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127524 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea8b7360-aead-4ea7-858c-047ce3c341b7-cni-binary-copy\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.127982 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127638 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-slash\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.127982 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127688 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-run-openvswitch\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.127982 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127728 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-cni-netd\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.127982 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.127733 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:16.127982 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127761 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/795198a1-531e-4263-89f7-59d4a963ef34-ovnkube-script-lib\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.127982 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127842 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ea8b7360-aead-4ea7-858c-047ce3c341b7-multus-daemon-config\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.127982 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127925 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/653a14cf-5131-4a36-8c13-35f8ab9d499a-cni-binary-copy\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.128477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.127993 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.128477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128034 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm7fg\" (UniqueName: \"kubernetes.io/projected/653a14cf-5131-4a36-8c13-35f8ab9d499a-kube-api-access-zm7fg\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.128477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128065 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-systemd-units\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.128477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128104 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-run-ovn\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.128477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128162 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8d49d458-d0eb-4619-9b41-2aa12b664541-dbus\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:16.128477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128187 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:16.128477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128208 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-cnibin\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.128477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128235 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-multus-conf-dir\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.128477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128262 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a67276d8-00ac-46dd-b864-cdd45f909348-iptables-alerter-script\") pod \"iptables-alerter-xxswm\" (UID: \"a67276d8-00ac-46dd-b864-cdd45f909348\") " pod="openshift-network-operator/iptables-alerter-xxswm" Sep 19 05:41:16.128477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128303 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-run-netns\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.128477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128351 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-run-multus-certs\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.128477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128448 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjwxh\" (UniqueName: \"kubernetes.io/projected/ea8b7360-aead-4ea7-858c-047ce3c341b7-kube-api-access-cjwxh\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.129055 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128512 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-device-dir\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.129055 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128555 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/653a14cf-5131-4a36-8c13-35f8ab9d499a-system-cni-dir\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.129055 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128585 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/653a14cf-5131-4a36-8c13-35f8ab9d499a-cnibin\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.129055 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128616 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8d49d458-d0eb-4619-9b41-2aa12b664541-kubelet-config\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:16.129055 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128646 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.129055 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128710 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-sys-fs\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.129055 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128832 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-etc-openvswitch\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.129055 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128884 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/795198a1-531e-4263-89f7-59d4a963ef34-ovn-node-metrics-cert\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.129055 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128910 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-etc-kubernetes\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.129055 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.128966 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-socket-dir\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.129528 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.129086 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a67276d8-00ac-46dd-b864-cdd45f909348-host-slash\") pod \"iptables-alerter-xxswm\" (UID: \"a67276d8-00ac-46dd-b864-cdd45f909348\") " pod="openshift-network-operator/iptables-alerter-xxswm" Sep 19 05:41:16.129528 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.129129 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-node-log\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.129528 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.129160 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5d68\" (UniqueName: \"kubernetes.io/projected/795198a1-531e-4263-89f7-59d4a963ef34-kube-api-access-l5d68\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.129528 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.129193 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-hostroot\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.130176 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.130154 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 19 05:41:16.130265 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.130193 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:16.130320 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.130268 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 19 05:41:16.130320 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.130271 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:16.130521 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.130379 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 19 05:41:16.130521 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.130511 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rc6pn\"" Sep 19 05:41:16.175745 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.175712 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dglxl" Sep 19 05:41:16.182876 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.182632 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dglxl" Sep 19 05:41:16.215126 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.215091 2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 19 05:41:16.230128 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230093 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/795198a1-531e-4263-89f7-59d4a963ef34-ovn-node-metrics-cert\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.230302 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230139 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-etc-kubernetes\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.230302 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230177 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c30c631d-63ed-410c-8d71-a0f4384bae54-host\") pod \"node-ca-5fm2w\" (UID: \"c30c631d-63ed-410c-8d71-a0f4384bae54\") " pod="openshift-image-registry/node-ca-5fm2w" Sep 19 05:41:16.230302 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230209 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-socket-dir\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.230302 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230235 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a67276d8-00ac-46dd-b864-cdd45f909348-host-slash\") pod \"iptables-alerter-xxswm\" (UID: \"a67276d8-00ac-46dd-b864-cdd45f909348\") " pod="openshift-network-operator/iptables-alerter-xxswm" Sep 19 05:41:16.230302 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230263 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-node-log\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.230302 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230287 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5d68\" (UniqueName: \"kubernetes.io/projected/795198a1-531e-4263-89f7-59d4a963ef34-kube-api-access-l5d68\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230311 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-hostroot\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230333 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-sysctl-conf\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230355 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-tuned\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230379 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-etc-selinux\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230419 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xfmz\" (UniqueName: \"kubernetes.io/projected/9355f974-6938-4adf-9c75-280336afb66f-kube-api-access-4xfmz\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230443 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78tgj\" (UniqueName: \"kubernetes.io/projected/11de77e7-d493-4516-be47-51e244b1c3fe-kube-api-access-78tgj\") pod \"node-resolver-mmcd6\" (UID: \"11de77e7-d493-4516-be47-51e244b1c3fe\") " pod="openshift-dns/node-resolver-mmcd6" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230468 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q48hm\" (UniqueName: \"kubernetes.io/projected/a67276d8-00ac-46dd-b864-cdd45f909348-kube-api-access-q48hm\") pod \"iptables-alerter-xxswm\" (UID: \"a67276d8-00ac-46dd-b864-cdd45f909348\") " pod="openshift-network-operator/iptables-alerter-xxswm" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230490 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-run-systemd\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230519 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-run-k8s-cni-cncf-io\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230542 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-systemd\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230565 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-host\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230591 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/653a14cf-5131-4a36-8c13-35f8ab9d499a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.230644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230643 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-log-socket\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230671 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njm9c\" (UniqueName: \"kubernetes.io/projected/ff709368-cadb-49c8-8bf1-eb0c3938922c-kube-api-access-njm9c\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230698 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-kubelet\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230727 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-run-netns\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230751 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-system-cni-dir\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230774 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-var-lib-cni-bin\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230799 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-var-lib-cni-multus\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230829 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vmtm\" (UniqueName: \"kubernetes.io/projected/c30c631d-63ed-410c-8d71-a0f4384bae54-kube-api-access-9vmtm\") pod \"node-ca-5fm2w\" (UID: \"c30c631d-63ed-410c-8d71-a0f4384bae54\") " pod="openshift-image-registry/node-ca-5fm2w" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230854 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/653a14cf-5131-4a36-8c13-35f8ab9d499a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230908 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-cni-bin\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230936 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-kubernetes\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230959 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-sys\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.230991 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-var-lib-kubelet\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231016 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-288vl\" (UniqueName: \"kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl\") pod \"network-check-target-pnp4s\" (UID: \"27fb3423-d426-4d0c-b117-439667e2be79\") " pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231043 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11de77e7-d493-4516-be47-51e244b1c3fe-hosts-file\") pod \"node-resolver-mmcd6\" (UID: \"11de77e7-d493-4516-be47-51e244b1c3fe\") " pod="openshift-dns/node-resolver-mmcd6" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231074 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-sysconfig\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231098 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mhxg\" (UniqueName: \"kubernetes.io/projected/1e522fc7-25ee-4b8b-912f-5cac3c092fab-kube-api-access-8mhxg\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.231256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231131 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231155 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f34c569d-7d66-4101-be51-00dbcc995a30-konnectivity-ca\") pod \"konnectivity-agent-7jfqn\" (UID: \"f34c569d-7d66-4101-be51-00dbcc995a30\") " pod="kube-system/konnectivity-agent-7jfqn" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231190 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-run-ovn-kubernetes\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231219 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-var-lib-kubelet\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231250 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-multus-socket-dir-parent\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231279 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/653a14cf-5131-4a36-8c13-35f8ab9d499a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231305 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/795198a1-531e-4263-89f7-59d4a963ef34-ovnkube-config\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231330 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11de77e7-d493-4516-be47-51e244b1c3fe-tmp-dir\") pod \"node-resolver-mmcd6\" (UID: \"11de77e7-d493-4516-be47-51e244b1c3fe\") " pod="openshift-dns/node-resolver-mmcd6" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231357 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-registration-dir\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231382 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-var-lib-openvswitch\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231420 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/795198a1-531e-4263-89f7-59d4a963ef34-env-overrides\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231447 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-multus-cni-dir\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231471 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea8b7360-aead-4ea7-858c-047ce3c341b7-cni-binary-copy\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231498 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-slash\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231523 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-run-openvswitch\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231550 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-cni-netd\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231575 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/795198a1-531e-4263-89f7-59d4a963ef34-ovnkube-script-lib\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.232112 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231598 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ea8b7360-aead-4ea7-858c-047ce3c341b7-multus-daemon-config\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231623 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/653a14cf-5131-4a36-8c13-35f8ab9d499a-cni-binary-copy\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231649 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231676 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm7fg\" (UniqueName: \"kubernetes.io/projected/653a14cf-5131-4a36-8c13-35f8ab9d499a-kube-api-access-zm7fg\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231702 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-systemd-units\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231726 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-run-ovn\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231751 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8d49d458-d0eb-4619-9b41-2aa12b664541-dbus\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231776 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231800 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-cnibin\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231824 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-multus-conf-dir\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231850 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-modprobe-d\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231876 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a67276d8-00ac-46dd-b864-cdd45f909348-iptables-alerter-script\") pod \"iptables-alerter-xxswm\" (UID: \"a67276d8-00ac-46dd-b864-cdd45f909348\") " pod="openshift-network-operator/iptables-alerter-xxswm" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231912 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-run-netns\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231938 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-run-multus-certs\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231963 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjwxh\" (UniqueName: \"kubernetes.io/projected/ea8b7360-aead-4ea7-858c-047ce3c341b7-kube-api-access-cjwxh\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.231994 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-sysctl-d\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232018 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-lib-modules\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.232726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232050 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e522fc7-25ee-4b8b-912f-5cac3c092fab-tmp\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232076 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-device-dir\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232105 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/653a14cf-5131-4a36-8c13-35f8ab9d499a-system-cni-dir\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232132 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/653a14cf-5131-4a36-8c13-35f8ab9d499a-cnibin\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232158 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8d49d458-d0eb-4619-9b41-2aa12b664541-kubelet-config\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232181 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-run\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232205 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f34c569d-7d66-4101-be51-00dbcc995a30-agent-certs\") pod \"konnectivity-agent-7jfqn\" (UID: \"f34c569d-7d66-4101-be51-00dbcc995a30\") " pod="kube-system/konnectivity-agent-7jfqn" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232231 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c30c631d-63ed-410c-8d71-a0f4384bae54-serviceca\") pod \"node-ca-5fm2w\" (UID: \"c30c631d-63ed-410c-8d71-a0f4384bae54\") " pod="openshift-image-registry/node-ca-5fm2w" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232258 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232284 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-sys-fs\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232342 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-etc-openvswitch\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232456 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-etc-openvswitch\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232847 2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232904 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-etc-kubernetes\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.232956 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-multus-socket-dir-parent\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233011 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-kubelet\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233055 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-run-netns\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233066 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-socket-dir\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.233247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233109 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-system-cni-dir\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233114 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a67276d8-00ac-46dd-b864-cdd45f909348-host-slash\") pod \"iptables-alerter-xxswm\" (UID: \"a67276d8-00ac-46dd-b864-cdd45f909348\") " pod="openshift-network-operator/iptables-alerter-xxswm" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233159 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-node-log\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233448 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-var-lib-cni-bin\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233463 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-cnibin\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233491 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-systemd-units\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233508 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-var-lib-cni-multus\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233523 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-multus-conf-dir\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233557 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-run-ovn\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233652 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/653a14cf-5131-4a36-8c13-35f8ab9d499a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233665 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8d49d458-d0eb-4619-9b41-2aa12b664541-dbus\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233690 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-cni-bin\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233736 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11de77e7-d493-4516-be47-51e244b1c3fe-hosts-file\") pod \"node-resolver-mmcd6\" (UID: \"11de77e7-d493-4516-be47-51e244b1c3fe\") " pod="openshift-dns/node-resolver-mmcd6" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.233826 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.233889 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret podName:8d49d458-d0eb-4619-9b41-2aa12b664541 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:16.733867386 +0000 UTC m=+4.727094918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret") pod "global-pull-secret-syncer-s9kmn" (UID: "8d49d458-d0eb-4619-9b41-2aa12b664541") : object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233927 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-run-ovn-kubernetes\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.233936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.233936 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/653a14cf-5131-4a36-8c13-35f8ab9d499a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.234658 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234180 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a67276d8-00ac-46dd-b864-cdd45f909348-iptables-alerter-script\") pod \"iptables-alerter-xxswm\" (UID: \"a67276d8-00ac-46dd-b864-cdd45f909348\") " pod="openshift-network-operator/iptables-alerter-xxswm" Sep 19 05:41:16.234658 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234359 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/653a14cf-5131-4a36-8c13-35f8ab9d499a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.234658 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234422 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-var-lib-kubelet\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.234658 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234460 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-hostroot\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.234658 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234456 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-run-netns\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.234658 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234499 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-run-multus-certs\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.234658 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234544 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/795198a1-531e-4263-89f7-59d4a963ef34-ovnkube-config\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.234658 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234550 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-device-dir\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.234658 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234567 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-etc-selinux\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.234658 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234578 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/653a14cf-5131-4a36-8c13-35f8ab9d499a-cnibin\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.234658 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234602 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-cni-netd\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.234658 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234617 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8d49d458-d0eb-4619-9b41-2aa12b664541-kubelet-config\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:16.235193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.234746 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.235193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235003 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-var-lib-openvswitch\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.235193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235010 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.235193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235054 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-registration-dir\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.235193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235068 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/795198a1-531e-4263-89f7-59d4a963ef34-ovnkube-script-lib\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.235193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235098 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/653a14cf-5131-4a36-8c13-35f8ab9d499a-system-cni-dir\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.235193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235111 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-run-systemd\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.235193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235147 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-host-run-k8s-cni-cncf-io\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.235193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235148 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea8b7360-aead-4ea7-858c-047ce3c341b7-multus-cni-dir\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.235193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235193 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-log-socket\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.235664 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235236 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9355f974-6938-4adf-9c75-280336afb66f-sys-fs\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.235664 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235280 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-host-slash\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.235664 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235313 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/795198a1-531e-4263-89f7-59d4a963ef34-run-openvswitch\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.235664 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235558 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ea8b7360-aead-4ea7-858c-047ce3c341b7-multus-daemon-config\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.235664 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235660 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea8b7360-aead-4ea7-858c-047ce3c341b7-cni-binary-copy\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.235898 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.235837 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/653a14cf-5131-4a36-8c13-35f8ab9d499a-cni-binary-copy\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.236160 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.236138 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/795198a1-531e-4263-89f7-59d4a963ef34-env-overrides\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.236262 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.236166 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11de77e7-d493-4516-be47-51e244b1c3fe-tmp-dir\") pod \"node-resolver-mmcd6\" (UID: \"11de77e7-d493-4516-be47-51e244b1c3fe\") " pod="openshift-dns/node-resolver-mmcd6" Sep 19 05:41:16.238865 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.238843 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/795198a1-531e-4263-89f7-59d4a963ef34-ovn-node-metrics-cert\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.243814 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.243783 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5d68\" (UniqueName: \"kubernetes.io/projected/795198a1-531e-4263-89f7-59d4a963ef34-kube-api-access-l5d68\") pod \"ovnkube-node-tzjr9\" (UID: \"795198a1-531e-4263-89f7-59d4a963ef34\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.248499 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.248027 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78tgj\" (UniqueName: \"kubernetes.io/projected/11de77e7-d493-4516-be47-51e244b1c3fe-kube-api-access-78tgj\") pod \"node-resolver-mmcd6\" (UID: \"11de77e7-d493-4516-be47-51e244b1c3fe\") " pod="openshift-dns/node-resolver-mmcd6" Sep 19 05:41:16.248499 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.248428 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjwxh\" (UniqueName: \"kubernetes.io/projected/ea8b7360-aead-4ea7-858c-047ce3c341b7-kube-api-access-cjwxh\") pod \"multus-7lg7k\" (UID: \"ea8b7360-aead-4ea7-858c-047ce3c341b7\") " pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.249304 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.249278 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xfmz\" (UniqueName: \"kubernetes.io/projected/9355f974-6938-4adf-9c75-280336afb66f-kube-api-access-4xfmz\") pod \"aws-ebs-csi-driver-node-sc7hm\" (UID: \"9355f974-6938-4adf-9c75-280336afb66f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.251111 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.251065 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q48hm\" (UniqueName: \"kubernetes.io/projected/a67276d8-00ac-46dd-b864-cdd45f909348-kube-api-access-q48hm\") pod \"iptables-alerter-xxswm\" (UID: \"a67276d8-00ac-46dd-b864-cdd45f909348\") " pod="openshift-network-operator/iptables-alerter-xxswm" Sep 19 05:41:16.251263 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.251243 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm7fg\" (UniqueName: \"kubernetes.io/projected/653a14cf-5131-4a36-8c13-35f8ab9d499a-kube-api-access-zm7fg\") pod \"multus-additional-cni-plugins-gss8z\" (UID: \"653a14cf-5131-4a36-8c13-35f8ab9d499a\") " pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.267121 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.267065 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" event={"ID":"a6fb336e699793f1bfd1aad113e3eb3a","Type":"ContainerStarted","Data":"0f0f2079a4d06908545c3fbe0440ce4d58cbf0de436cb9450f6110195fa5edd7"} Sep 19 05:41:16.269865 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.269837 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-57.ec2.internal" event={"ID":"f782f89f5d636e4550fa0cef2186e194","Type":"ContainerStarted","Data":"6c884d136f394373337cc5056bbf5d42ae99fa7ecca2a4ce6a6fc561b394a406"} Sep 19 05:41:16.333431 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333374 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-modprobe-d\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.333634 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333440 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-sysctl-d\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.333634 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333466 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-lib-modules\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.333634 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333489 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e522fc7-25ee-4b8b-912f-5cac3c092fab-tmp\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.333634 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333514 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-run\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.333634 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333535 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f34c569d-7d66-4101-be51-00dbcc995a30-agent-certs\") pod \"konnectivity-agent-7jfqn\" (UID: \"f34c569d-7d66-4101-be51-00dbcc995a30\") " pod="kube-system/konnectivity-agent-7jfqn" Sep 19 05:41:16.333634 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333557 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c30c631d-63ed-410c-8d71-a0f4384bae54-serviceca\") pod \"node-ca-5fm2w\" (UID: \"c30c631d-63ed-410c-8d71-a0f4384bae54\") " pod="openshift-image-registry/node-ca-5fm2w" Sep 19 05:41:16.333634 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333579 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c30c631d-63ed-410c-8d71-a0f4384bae54-host\") pod \"node-ca-5fm2w\" (UID: \"c30c631d-63ed-410c-8d71-a0f4384bae54\") " pod="openshift-image-registry/node-ca-5fm2w" Sep 19 05:41:16.333634 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333604 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-sysctl-conf\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333633 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-tuned\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333700 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-systemd\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333720 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-host\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333746 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njm9c\" (UniqueName: \"kubernetes.io/projected/ff709368-cadb-49c8-8bf1-eb0c3938922c-kube-api-access-njm9c\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333775 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vmtm\" (UniqueName: \"kubernetes.io/projected/c30c631d-63ed-410c-8d71-a0f4384bae54-kube-api-access-9vmtm\") pod \"node-ca-5fm2w\" (UID: \"c30c631d-63ed-410c-8d71-a0f4384bae54\") " pod="openshift-image-registry/node-ca-5fm2w" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333800 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-kubernetes\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333820 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-sys\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333842 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-var-lib-kubelet\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333863 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288vl\" (UniqueName: \"kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl\") pod \"network-check-target-pnp4s\" (UID: \"27fb3423-d426-4d0c-b117-439667e2be79\") " pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333887 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-sysconfig\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333909 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mhxg\" (UniqueName: \"kubernetes.io/projected/1e522fc7-25ee-4b8b-912f-5cac3c092fab-kube-api-access-8mhxg\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.333931 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:16.334022 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.334025 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:16.334672 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.334060 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-sysctl-conf\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334672 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.334084 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs podName:ff709368-cadb-49c8-8bf1-eb0c3938922c nodeName:}" failed. No retries permitted until 2026-09-19 05:41:16.834065793 +0000 UTC m=+4.827293304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs") pod "network-metrics-daemon-8947n" (UID: "ff709368-cadb-49c8-8bf1-eb0c3938922c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:16.334672 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.334583 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c30c631d-63ed-410c-8d71-a0f4384bae54-serviceca\") pod \"node-ca-5fm2w\" (UID: \"c30c631d-63ed-410c-8d71-a0f4384bae54\") " pod="openshift-image-registry/node-ca-5fm2w" Sep 19 05:41:16.334672 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.334642 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c30c631d-63ed-410c-8d71-a0f4384bae54-host\") pod \"node-ca-5fm2w\" (UID: \"c30c631d-63ed-410c-8d71-a0f4384bae54\") " pod="openshift-image-registry/node-ca-5fm2w" Sep 19 05:41:16.334672 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.334633 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-run\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334926 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.334683 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-sys\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334926 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.334726 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-systemd\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334926 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.334761 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-host\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334926 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.334788 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-sysctl-d\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.334926 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.334863 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-modprobe-d\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.335157 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.334971 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-lib-modules\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.335157 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.335027 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-kubernetes\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.335157 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.335079 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-sysconfig\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.335157 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.335129 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e522fc7-25ee-4b8b-912f-5cac3c092fab-var-lib-kubelet\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.335387 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.335232 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f34c569d-7d66-4101-be51-00dbcc995a30-konnectivity-ca\") pod \"konnectivity-agent-7jfqn\" (UID: \"f34c569d-7d66-4101-be51-00dbcc995a30\") " pod="kube-system/konnectivity-agent-7jfqn" Sep 19 05:41:16.335820 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.335793 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f34c569d-7d66-4101-be51-00dbcc995a30-konnectivity-ca\") pod \"konnectivity-agent-7jfqn\" (UID: \"f34c569d-7d66-4101-be51-00dbcc995a30\") " pod="kube-system/konnectivity-agent-7jfqn" Sep 19 05:41:16.336672 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.336648 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e522fc7-25ee-4b8b-912f-5cac3c092fab-etc-tuned\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.336990 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.336973 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f34c569d-7d66-4101-be51-00dbcc995a30-agent-certs\") pod \"konnectivity-agent-7jfqn\" (UID: \"f34c569d-7d66-4101-be51-00dbcc995a30\") " pod="kube-system/konnectivity-agent-7jfqn" Sep 19 05:41:16.338203 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.338180 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e522fc7-25ee-4b8b-912f-5cac3c092fab-tmp\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.341310 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.341287 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 05:41:16.341310 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.341309 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 05:41:16.341497 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.341325 2584 projected.go:196] Error preparing data for projected volume kube-api-access-288vl for pod openshift-network-diagnostics/network-check-target-pnp4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:16.341497 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.341416 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl podName:27fb3423-d426-4d0c-b117-439667e2be79 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:16.841383463 +0000 UTC m=+4.834610986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-288vl" (UniqueName: "kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl") pod "network-check-target-pnp4s" (UID: "27fb3423-d426-4d0c-b117-439667e2be79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:16.344225 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.344203 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vmtm\" (UniqueName: \"kubernetes.io/projected/c30c631d-63ed-410c-8d71-a0f4384bae54-kube-api-access-9vmtm\") pod \"node-ca-5fm2w\" (UID: \"c30c631d-63ed-410c-8d71-a0f4384bae54\") " pod="openshift-image-registry/node-ca-5fm2w" Sep 19 05:41:16.344975 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.344954 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mhxg\" (UniqueName: \"kubernetes.io/projected/1e522fc7-25ee-4b8b-912f-5cac3c092fab-kube-api-access-8mhxg\") pod \"tuned-84zqx\" (UID: \"1e522fc7-25ee-4b8b-912f-5cac3c092fab\") " pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.344975 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.344966 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njm9c\" (UniqueName: \"kubernetes.io/projected/ff709368-cadb-49c8-8bf1-eb0c3938922c-kube-api-access-njm9c\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:16.413706 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.413625 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" Sep 19 05:41:16.425054 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:16.425027 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9355f974_6938_4adf_9c75_280336afb66f.slice/crio-a279f3b929ea4c33d451bafdcbf567edbc291bf9641dcbb82901e55ba5b9db6a WatchSource:0}: Error finding container a279f3b929ea4c33d451bafdcbf567edbc291bf9641dcbb82901e55ba5b9db6a: Status 404 returned error can't find the container with id a279f3b929ea4c33d451bafdcbf567edbc291bf9641dcbb82901e55ba5b9db6a Sep 19 05:41:16.432849 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.432826 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7lg7k" Sep 19 05:41:16.439321 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.439294 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xxswm" Sep 19 05:41:16.442138 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:16.442107 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8b7360_aead_4ea7_858c_047ce3c341b7.slice/crio-3aa3c0bca26652f481fb564698d0dd86ce00c1b33d002e4a9263b3351e86dbb2 WatchSource:0}: Error finding container 3aa3c0bca26652f481fb564698d0dd86ce00c1b33d002e4a9263b3351e86dbb2: Status 404 returned error can't find the container with id 3aa3c0bca26652f481fb564698d0dd86ce00c1b33d002e4a9263b3351e86dbb2 Sep 19 05:41:16.446818 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.446794 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gss8z" Sep 19 05:41:16.448832 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:16.448809 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda67276d8_00ac_46dd_b864_cdd45f909348.slice/crio-e1efcefe4ec2eabb5be7dc07e102e52e2fccff9c9a1cbe3af97975b5e45ddf67 WatchSource:0}: Error finding container e1efcefe4ec2eabb5be7dc07e102e52e2fccff9c9a1cbe3af97975b5e45ddf67: Status 404 returned error can't find the container with id e1efcefe4ec2eabb5be7dc07e102e52e2fccff9c9a1cbe3af97975b5e45ddf67 Sep 19 05:41:16.455423 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.455278 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:16.458202 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:16.458173 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod653a14cf_5131_4a36_8c13_35f8ab9d499a.slice/crio-90e42afb2c2b27f35ea4bf9021be113bd336c61e8f3614801dd06a23212e9907 WatchSource:0}: Error finding container 90e42afb2c2b27f35ea4bf9021be113bd336c61e8f3614801dd06a23212e9907: Status 404 returned error can't find the container with id 90e42afb2c2b27f35ea4bf9021be113bd336c61e8f3614801dd06a23212e9907 Sep 19 05:41:16.464288 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.464218 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mmcd6" Sep 19 05:41:16.464838 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:16.464815 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795198a1_531e_4263_89f7_59d4a963ef34.slice/crio-2e23f55910d15c0b314bde9dbabb24d0c0132b1c5aaef37763e5d97a0cd69397 WatchSource:0}: Error finding container 2e23f55910d15c0b314bde9dbabb24d0c0132b1c5aaef37763e5d97a0cd69397: Status 404 returned error can't find the container with id 2e23f55910d15c0b314bde9dbabb24d0c0132b1c5aaef37763e5d97a0cd69397 Sep 19 05:41:16.471725 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.471701 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7jfqn" Sep 19 05:41:16.480531 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.480464 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5fm2w" Sep 19 05:41:16.481596 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:16.481557 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf34c569d_7d66_4101_be51_00dbcc995a30.slice/crio-b5f7ad144124dfa67481987221cff940c505f39afb71ab02c74d4f75d859dd65 WatchSource:0}: Error finding container b5f7ad144124dfa67481987221cff940c505f39afb71ab02c74d4f75d859dd65: Status 404 returned error can't find the container with id b5f7ad144124dfa67481987221cff940c505f39afb71ab02c74d4f75d859dd65 Sep 19 05:41:16.485882 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.485848 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-84zqx" Sep 19 05:41:16.493368 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:16.493342 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30c631d_63ed_410c_8d71_a0f4384bae54.slice/crio-62b52c118456ea45541f8f33d5e12db98df2f3d52a09baefbdad9f9fbf6dbe0d WatchSource:0}: Error finding container 62b52c118456ea45541f8f33d5e12db98df2f3d52a09baefbdad9f9fbf6dbe0d: Status 404 returned error can't find the container with id 62b52c118456ea45541f8f33d5e12db98df2f3d52a09baefbdad9f9fbf6dbe0d Sep 19 05:41:16.496657 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:41:16.496626 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e522fc7_25ee_4b8b_912f_5cac3c092fab.slice/crio-bfd11ea55fdcd76eddaa9ad6f2db0dc3cc7ab2c229e0236f39410085515f67ad WatchSource:0}: Error finding container bfd11ea55fdcd76eddaa9ad6f2db0dc3cc7ab2c229e0236f39410085515f67ad: Status 404 returned error can't find the container with id bfd11ea55fdcd76eddaa9ad6f2db0dc3cc7ab2c229e0236f39410085515f67ad Sep 19 05:41:16.738287 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.738194 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:16.738460 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.738353 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:16.738460 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.738431 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret podName:8d49d458-d0eb-4619-9b41-2aa12b664541 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:17.738414218 +0000 UTC m=+5.731641748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret") pod "global-pull-secret-syncer-s9kmn" (UID: "8d49d458-d0eb-4619-9b41-2aa12b664541") : object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:16.839215 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.839180 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:16.839373 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.839346 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:16.839442 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.839431 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs podName:ff709368-cadb-49c8-8bf1-eb0c3938922c nodeName:}" failed. No retries permitted until 2026-09-19 05:41:17.839415237 +0000 UTC m=+5.832642759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs") pod "network-metrics-daemon-8947n" (UID: "ff709368-cadb-49c8-8bf1-eb0c3938922c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:16.940130 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:16.940089 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288vl\" (UniqueName: \"kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl\") pod \"network-check-target-pnp4s\" (UID: \"27fb3423-d426-4d0c-b117-439667e2be79\") " pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:16.940302 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.940260 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 05:41:16.940302 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.940289 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 05:41:16.940302 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.940302 2584 projected.go:196] Error preparing data for projected volume kube-api-access-288vl for pod openshift-network-diagnostics/network-check-target-pnp4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:16.940500 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:16.940362 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl podName:27fb3423-d426-4d0c-b117-439667e2be79 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:17.940348717 +0000 UTC m=+5.933576226 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-288vl" (UniqueName: "kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl") pod "network-check-target-pnp4s" (UID: "27fb3423-d426-4d0c-b117-439667e2be79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:17.185806 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.185709 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 05:36:16 +0000 UTC" deadline="2028-03-31 00:31:18.911766373 +0000 UTC" Sep 19 05:41:17.185806 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.185758 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13410h50m1.72601394s" Sep 19 05:41:17.260853 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.260813 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:17.261032 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:17.260937 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:17.273587 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.273549 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5fm2w" event={"ID":"c30c631d-63ed-410c-8d71-a0f4384bae54","Type":"ContainerStarted","Data":"62b52c118456ea45541f8f33d5e12db98df2f3d52a09baefbdad9f9fbf6dbe0d"} Sep 19 05:41:17.275430 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.275100 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7jfqn" event={"ID":"f34c569d-7d66-4101-be51-00dbcc995a30","Type":"ContainerStarted","Data":"b5f7ad144124dfa67481987221cff940c505f39afb71ab02c74d4f75d859dd65"} Sep 19 05:41:17.276696 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.276643 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gss8z" event={"ID":"653a14cf-5131-4a36-8c13-35f8ab9d499a","Type":"ContainerStarted","Data":"90e42afb2c2b27f35ea4bf9021be113bd336c61e8f3614801dd06a23212e9907"} Sep 19 05:41:17.278178 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.278153 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-84zqx" event={"ID":"1e522fc7-25ee-4b8b-912f-5cac3c092fab","Type":"ContainerStarted","Data":"bfd11ea55fdcd76eddaa9ad6f2db0dc3cc7ab2c229e0236f39410085515f67ad"} Sep 19 05:41:17.279481 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.279438 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mmcd6" event={"ID":"11de77e7-d493-4516-be47-51e244b1c3fe","Type":"ContainerStarted","Data":"12d52e56a629a87e5d77617cd504da3fa9922060ddd55a029ae122e6c1b25a4a"} Sep 19 05:41:17.281148 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.281117 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" event={"ID":"795198a1-531e-4263-89f7-59d4a963ef34","Type":"ContainerStarted","Data":"2e23f55910d15c0b314bde9dbabb24d0c0132b1c5aaef37763e5d97a0cd69397"} Sep 19 05:41:17.282723 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.282698 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xxswm" event={"ID":"a67276d8-00ac-46dd-b864-cdd45f909348","Type":"ContainerStarted","Data":"e1efcefe4ec2eabb5be7dc07e102e52e2fccff9c9a1cbe3af97975b5e45ddf67"} Sep 19 05:41:17.284193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.284169 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7lg7k" event={"ID":"ea8b7360-aead-4ea7-858c-047ce3c341b7","Type":"ContainerStarted","Data":"3aa3c0bca26652f481fb564698d0dd86ce00c1b33d002e4a9263b3351e86dbb2"} Sep 19 05:41:17.286106 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.286080 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" event={"ID":"9355f974-6938-4adf-9c75-280336afb66f","Type":"ContainerStarted","Data":"a279f3b929ea4c33d451bafdcbf567edbc291bf9641dcbb82901e55ba5b9db6a"} Sep 19 05:41:17.747478 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.746849 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:17.747478 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:17.747054 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:17.747478 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:17.747126 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret podName:8d49d458-d0eb-4619-9b41-2aa12b664541 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:19.747107698 +0000 UTC m=+7.740335210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret") pod "global-pull-secret-syncer-s9kmn" (UID: "8d49d458-d0eb-4619-9b41-2aa12b664541") : object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:17.848010 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.847422 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:17.848010 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:17.847584 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:17.848010 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:17.847652 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs podName:ff709368-cadb-49c8-8bf1-eb0c3938922c nodeName:}" failed. No retries permitted until 2026-09-19 05:41:19.847633235 +0000 UTC m=+7.840860746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs") pod "network-metrics-daemon-8947n" (UID: "ff709368-cadb-49c8-8bf1-eb0c3938922c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:17.951457 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:17.948250 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288vl\" (UniqueName: \"kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl\") pod \"network-check-target-pnp4s\" (UID: \"27fb3423-d426-4d0c-b117-439667e2be79\") " pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:17.951705 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:17.951669 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 05:41:17.951784 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:17.951708 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 05:41:17.951784 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:17.951723 2584 projected.go:196] Error preparing data for projected volume kube-api-access-288vl for pod openshift-network-diagnostics/network-check-target-pnp4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:17.951884 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:17.951801 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl podName:27fb3423-d426-4d0c-b117-439667e2be79 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:19.951781179 +0000 UTC m=+7.945008705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-288vl" (UniqueName: "kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl") pod "network-check-target-pnp4s" (UID: "27fb3423-d426-4d0c-b117-439667e2be79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:18.186156 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:18.185843 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 05:36:16 +0000 UTC" deadline="2028-05-09 02:03:27.294164294 +0000 UTC" Sep 19 05:41:18.186156 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:18.186123 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14348h22m9.108046765s" Sep 19 05:41:18.263738 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:18.263707 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:18.263927 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:18.263868 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:18.264303 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:18.264282 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:18.264444 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:18.264391 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:18.306322 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:18.306226 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-57.ec2.internal" event={"ID":"f782f89f5d636e4550fa0cef2186e194","Type":"ContainerStarted","Data":"c37eefbf31dbcc1e5bbfb6b3afd4d25ef761250812c77051501d19ac1c1881c5"} Sep 19 05:41:19.260856 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:19.260817 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:19.261343 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:19.260955 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:19.315302 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:19.315258 2584 generic.go:356] "Generic (PLEG): container finished" podID="a6fb336e699793f1bfd1aad113e3eb3a" containerID="d9f923ba6c0b32ee1f04823890cf56a075563a012e48bebd325e991862ec9ad4" exitCode=0 Sep 19 05:41:19.316015 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:19.315848 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" event={"ID":"a6fb336e699793f1bfd1aad113e3eb3a","Type":"ContainerDied","Data":"d9f923ba6c0b32ee1f04823890cf56a075563a012e48bebd325e991862ec9ad4"} Sep 19 05:41:19.343463 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:19.343383 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-57.ec2.internal" podStartSLOduration=4.343363749 podStartE2EDuration="4.343363749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:18.323463422 +0000 UTC m=+6.316690954" watchObservedRunningTime="2026-09-19 05:41:19.343363749 +0000 UTC m=+7.336591282" Sep 19 05:41:19.768866 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:19.768179 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:19.768866 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:19.768386 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:19.768866 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:19.768501 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret podName:8d49d458-d0eb-4619-9b41-2aa12b664541 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:23.768472332 +0000 UTC m=+11.761699847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret") pod "global-pull-secret-syncer-s9kmn" (UID: "8d49d458-d0eb-4619-9b41-2aa12b664541") : object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:19.869457 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:19.869180 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:19.870374 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:19.869843 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:19.870374 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:19.869928 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs podName:ff709368-cadb-49c8-8bf1-eb0c3938922c nodeName:}" failed. No retries permitted until 2026-09-19 05:41:23.86990715 +0000 UTC m=+11.863134663 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs") pod "network-metrics-daemon-8947n" (UID: "ff709368-cadb-49c8-8bf1-eb0c3938922c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:19.969912 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:19.969880 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288vl\" (UniqueName: \"kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl\") pod \"network-check-target-pnp4s\" (UID: \"27fb3423-d426-4d0c-b117-439667e2be79\") " pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:19.970115 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:19.970082 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 05:41:19.970115 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:19.970103 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 05:41:19.970115 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:19.970116 2584 projected.go:196] Error preparing data for projected volume kube-api-access-288vl for pod openshift-network-diagnostics/network-check-target-pnp4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:19.970286 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:19.970170 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl podName:27fb3423-d426-4d0c-b117-439667e2be79 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:23.970152013 +0000 UTC m=+11.963379526 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-288vl" (UniqueName: "kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl") pod "network-check-target-pnp4s" (UID: "27fb3423-d426-4d0c-b117-439667e2be79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:20.263732 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:20.262975 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:20.263732 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:20.263104 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:20.263732 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:20.263493 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:20.263732 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:20.263604 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:21.260996 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:21.260960 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:21.261190 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:21.261094 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:22.260862 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:22.260822 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:22.261248 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:22.260956 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:22.261359 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:22.261334 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:22.261510 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:22.261485 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:23.260525 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:23.260382 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:23.260713 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:23.260548 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:23.803833 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:23.803793 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:23.804289 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:23.803934 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:23.804289 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:23.804003 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret podName:8d49d458-d0eb-4619-9b41-2aa12b664541 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:31.803982826 +0000 UTC m=+19.797210338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret") pod "global-pull-secret-syncer-s9kmn" (UID: "8d49d458-d0eb-4619-9b41-2aa12b664541") : object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:23.904323 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:23.904290 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:23.904512 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:23.904464 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:23.904561 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:23.904551 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs podName:ff709368-cadb-49c8-8bf1-eb0c3938922c nodeName:}" failed. No retries permitted until 2026-09-19 05:41:31.904532274 +0000 UTC m=+19.897759803 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs") pod "network-metrics-daemon-8947n" (UID: "ff709368-cadb-49c8-8bf1-eb0c3938922c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:24.005336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:24.005284 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288vl\" (UniqueName: \"kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl\") pod \"network-check-target-pnp4s\" (UID: \"27fb3423-d426-4d0c-b117-439667e2be79\") " pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:24.005529 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:24.005504 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 05:41:24.005529 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:24.005528 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 05:41:24.005620 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:24.005542 2584 projected.go:196] Error preparing data for projected volume kube-api-access-288vl for pod openshift-network-diagnostics/network-check-target-pnp4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:24.005620 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:24.005608 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl podName:27fb3423-d426-4d0c-b117-439667e2be79 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:32.005590281 +0000 UTC m=+19.998817796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-288vl" (UniqueName: "kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl") pod "network-check-target-pnp4s" (UID: "27fb3423-d426-4d0c-b117-439667e2be79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:24.261428 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:24.260659 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:24.261428 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:24.260789 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:24.262511 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:24.262473 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:24.262689 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:24.262644 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:25.260903 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:25.260804 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:25.261642 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:25.261007 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:26.260743 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:26.260702 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:26.260955 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:26.260854 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:26.260955 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:26.260906 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:26.261368 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:26.261002 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:27.260950 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:27.260866 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:27.261351 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:27.261003 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:28.260986 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:28.260947 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:28.260986 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:28.260970 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:28.261487 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:28.261093 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:28.261487 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:28.261217 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:29.260461 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:29.260421 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:29.260664 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:29.260543 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:30.261338 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:30.261298 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:30.261799 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:30.261445 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:30.261799 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:30.261485 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:30.261799 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:30.261611 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:31.261164 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:31.261124 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:31.261333 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:31.261257 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:31.866432 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:31.866383 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:31.866865 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:31.866559 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:31.866865 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:31.866628 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret podName:8d49d458-d0eb-4619-9b41-2aa12b664541 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:47.866612408 +0000 UTC m=+35.859839916 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret") pod "global-pull-secret-syncer-s9kmn" (UID: "8d49d458-d0eb-4619-9b41-2aa12b664541") : object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:31.967513 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:31.967461 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:31.967681 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:31.967634 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:31.967739 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:31.967715 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs podName:ff709368-cadb-49c8-8bf1-eb0c3938922c nodeName:}" failed. No retries permitted until 2026-09-19 05:41:47.967693883 +0000 UTC m=+35.960921395 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs") pod "network-metrics-daemon-8947n" (UID: "ff709368-cadb-49c8-8bf1-eb0c3938922c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:32.068752 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:32.068705 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288vl\" (UniqueName: \"kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl\") pod \"network-check-target-pnp4s\" (UID: \"27fb3423-d426-4d0c-b117-439667e2be79\") " pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:32.068943 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:32.068916 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 05:41:32.069009 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:32.068963 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 05:41:32.069009 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:32.068979 2584 projected.go:196] Error preparing data for projected volume kube-api-access-288vl for pod openshift-network-diagnostics/network-check-target-pnp4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:32.069078 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:32.069045 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl podName:27fb3423-d426-4d0c-b117-439667e2be79 nodeName:}" failed. No retries permitted until 2026-09-19 05:41:48.069025941 +0000 UTC m=+36.062253455 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-288vl" (UniqueName: "kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl") pod "network-check-target-pnp4s" (UID: "27fb3423-d426-4d0c-b117-439667e2be79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:32.261220 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:32.261127 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:32.261220 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:32.261174 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:32.261459 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:32.261288 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:32.261459 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:32.261410 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:33.261007 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:33.260967 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:33.261410 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:33.261075 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:34.261767 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:34.261735 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:34.262111 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:34.261820 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:34.262111 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:34.261873 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:34.262111 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:34.261962 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:35.261458 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.260994 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:35.261592 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:35.261532 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:35.349913 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.349870 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" event={"ID":"a6fb336e699793f1bfd1aad113e3eb3a","Type":"ContainerStarted","Data":"4d596fb48b05d52b2abe493979feaa04a79ff0c7afd9db3a8565a9dec75243f9"} Sep 19 05:41:35.351771 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.351742 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5fm2w" event={"ID":"c30c631d-63ed-410c-8d71-a0f4384bae54","Type":"ContainerStarted","Data":"4e1989e464b969b852e1af5d3ccc299f4facd4216b87989db8da5e205faccc55"} Sep 19 05:41:35.353600 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.353509 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7jfqn" event={"ID":"f34c569d-7d66-4101-be51-00dbcc995a30","Type":"ContainerStarted","Data":"7238a867ee696ca936958c74029acdb8d0a600fedcff51d14eec861e2e1c0273"} Sep 19 05:41:35.355198 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.355173 2584 generic.go:356] "Generic (PLEG): container finished" podID="653a14cf-5131-4a36-8c13-35f8ab9d499a" containerID="64755ac31d3da8dad1896900b8cfda129be0f98c63df2e05e42a7de963b5a3c2" exitCode=0 Sep 19 05:41:35.355305 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.355263 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gss8z" event={"ID":"653a14cf-5131-4a36-8c13-35f8ab9d499a","Type":"ContainerDied","Data":"64755ac31d3da8dad1896900b8cfda129be0f98c63df2e05e42a7de963b5a3c2"} Sep 19 05:41:35.356841 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.356819 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-84zqx" event={"ID":"1e522fc7-25ee-4b8b-912f-5cac3c092fab","Type":"ContainerStarted","Data":"55a66a1190ea3574bab02695029b40f3c1df0f81070e74a0ccf8b8569e7d77f6"} Sep 19 05:41:35.358374 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.358354 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mmcd6" event={"ID":"11de77e7-d493-4516-be47-51e244b1c3fe","Type":"ContainerStarted","Data":"9deec4ae0cd7b70c4b33089e0f8cb1c361677f9178cc7f69b96ad3cea6ea2da6"} Sep 19 05:41:35.360943 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.360927 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 05:41:35.361247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.361226 2584 generic.go:356] "Generic (PLEG): container finished" podID="795198a1-531e-4263-89f7-59d4a963ef34" containerID="7a17df0dc9718e79c25cca4fddc23b2e5ebd15f0ab7042f3946514a8d3eaeb19" exitCode=1 Sep 19 05:41:35.361324 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.361290 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" event={"ID":"795198a1-531e-4263-89f7-59d4a963ef34","Type":"ContainerStarted","Data":"6b8a6d17cf78a93a78a932b16bee5eb5149b7c4c33e4e521c4f73e3a18dd15b2"} Sep 19 05:41:35.361371 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.361352 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" event={"ID":"795198a1-531e-4263-89f7-59d4a963ef34","Type":"ContainerStarted","Data":"e564cd8372c271709e6801e7e870eb45971d2210556ccf480d13f57f215e59f4"} Sep 19 05:41:35.361433 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.361386 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" event={"ID":"795198a1-531e-4263-89f7-59d4a963ef34","Type":"ContainerStarted","Data":"9ffb4aed6146160d23dbae98286c83a92331b7fb46185825bbde15cc62e6eb2b"} Sep 19 05:41:35.361473 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.361433 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" event={"ID":"795198a1-531e-4263-89f7-59d4a963ef34","Type":"ContainerStarted","Data":"2be446c3963f26c8f5616fedf598eec0898a59069be6fc5ed4077debadb6e1b3"} Sep 19 05:41:35.361473 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.361466 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" event={"ID":"795198a1-531e-4263-89f7-59d4a963ef34","Type":"ContainerDied","Data":"7a17df0dc9718e79c25cca4fddc23b2e5ebd15f0ab7042f3946514a8d3eaeb19"} Sep 19 05:41:35.361565 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.361501 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" event={"ID":"795198a1-531e-4263-89f7-59d4a963ef34","Type":"ContainerStarted","Data":"fdd9ed7d9257c1dda19600b8c726b3fa2e22e175c65030c751cefb0941708e64"} Sep 19 05:41:35.362660 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.362643 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7lg7k" event={"ID":"ea8b7360-aead-4ea7-858c-047ce3c341b7","Type":"ContainerStarted","Data":"5a0805ee426ff80e7a03784c4e610c67e59b156ca8b79b2b1af41edfa6173e17"} Sep 19 05:41:35.363918 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.363900 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" event={"ID":"9355f974-6938-4adf-9c75-280336afb66f","Type":"ContainerStarted","Data":"cd8730d8524a9eb570af42fa09a00e129e0f3abb0af0b708f37fb2911ccbaa4b"} Sep 19 05:41:35.369620 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.369576 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-57.ec2.internal" podStartSLOduration=20.369561128 podStartE2EDuration="20.369561128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:35.369391482 +0000 UTC m=+23.362619013" watchObservedRunningTime="2026-09-19 05:41:35.369561128 +0000 UTC m=+23.362788659" Sep 19 05:41:35.407735 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.407685 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7jfqn" podStartSLOduration=7.398407819 podStartE2EDuration="21.407667913s" totalImagesPullingTime="14.009260094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:35.407118832 +0000 UTC m=+23.400346360" watchObservedRunningTime="2026-09-19 05:41:35.407667913 +0000 UTC m=+23.400895445" Sep 19 05:41:35.427095 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.427038 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-84zqx" podStartSLOduration=3.56892635 podStartE2EDuration="21.42702186s" totalImagesPullingTime="17.85809551s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:35.426522757 +0000 UTC m=+23.419750288" watchObservedRunningTime="2026-09-19 05:41:35.42702186 +0000 UTC m=+23.420249430" Sep 19 05:41:35.446318 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.446267 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mmcd6" podStartSLOduration=3.596998708 podStartE2EDuration="21.446252774s" totalImagesPullingTime="17.849254066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:35.446211382 +0000 UTC m=+23.439438912" watchObservedRunningTime="2026-09-19 05:41:35.446252774 +0000 UTC m=+23.439480304" Sep 19 05:41:35.465645 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.465595 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5fm2w" podStartSLOduration=3.6346175819999997 podStartE2EDuration="21.465579179s" totalImagesPullingTime="17.830961597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:35.464991347 +0000 UTC m=+23.458218876" watchObservedRunningTime="2026-09-19 05:41:35.465579179 +0000 UTC m=+23.458806709" Sep 19 05:41:35.484226 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.484168 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7lg7k" podStartSLOduration=3.570651795 podStartE2EDuration="21.4841493s" totalImagesPullingTime="17.913497505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:35.484014293 +0000 UTC m=+23.477241829" watchObservedRunningTime="2026-09-19 05:41:35.4841493 +0000 UTC m=+23.477376832" Sep 19 05:41:35.650304 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.650220 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:35.721353 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.721313 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:35.882517 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.882477 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:35.982549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:35.982515 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:36.055327 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:36.055297 2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 19 05:41:36.199883 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:36.199702 2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T05:41:36.055318624Z","UUID":"ce74b4f6-6d68-4527-b1a1-45f9486d481e","Handler":null,"Name":"","Endpoint":""} Sep 19 05:41:36.202041 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:36.202010 2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 19 05:41:36.202170 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:36.202091 2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 19 05:41:36.261470 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:36.261437 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:36.261635 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:36.261437 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:36.261635 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:36.261581 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:36.261765 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:36.261641 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:36.367655 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:36.367615 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xxswm" event={"ID":"a67276d8-00ac-46dd-b864-cdd45f909348","Type":"ContainerStarted","Data":"7ef110e6e08257005770d57714846b8f0a68251942dec30f2d9a9bf73d03c054"} Sep 19 05:41:36.369768 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:36.369724 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" event={"ID":"9355f974-6938-4adf-9c75-280336afb66f","Type":"ContainerStarted","Data":"2935e37e524a836ba0d4b99a2d2f5c3b5148a15b8970d20cd04a6c7d37375b74"} Sep 19 05:41:36.637225 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:36.637139 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:36.727905 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:36.727871 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:36.972726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:36.972637 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:37.106344 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.106312 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:37.261268 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.261234 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:37.261434 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:37.261365 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:37.377113 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.377081 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 05:41:37.377596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.377521 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" event={"ID":"795198a1-531e-4263-89f7-59d4a963ef34","Type":"ContainerStarted","Data":"9dae134fb5a6b307dbce12c554a04e40b132998f4ad2cdd8230364103de3bd2d"} Sep 19 05:41:37.380220 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.380186 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" event={"ID":"9355f974-6938-4adf-9c75-280336afb66f","Type":"ContainerStarted","Data":"9c8a6e144506f30a0cd19cd5e22ddc0027c19f497db1db28a44f425697dbbd34"} Sep 19 05:41:37.384767 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.384690 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:37.404738 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.404681 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xxswm" podStartSLOduration=5.528950326 podStartE2EDuration="23.404665229s" totalImagesPullingTime="17.875714903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:36.39433194 +0000 UTC m=+24.387559465" watchObservedRunningTime="2026-09-19 05:41:37.404665229 +0000 UTC m=+25.397892764" Sep 19 05:41:37.485229 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.485199 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:37.511305 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.511227 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7jfqn" Sep 19 05:41:37.511793 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.511776 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7jfqn" Sep 19 05:41:37.527619 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.527568 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sc7hm" podStartSLOduration=7.050589244 podStartE2EDuration="23.527547978s" totalImagesPullingTime="16.476958734s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:37.404601713 +0000 UTC m=+25.397829247" watchObservedRunningTime="2026-09-19 05:41:37.527547978 +0000 UTC m=+25.520775511" Sep 19 05:41:37.654336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.654139 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:37.741947 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:37.741911 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:38.126875 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:38.126852 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mmcd6_11de77e7-d493-4516-be47-51e244b1c3fe/dns-node-resolver/0.log" Sep 19 05:41:38.261178 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:38.261143 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:38.261178 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:38.261179 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:38.261440 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:38.261278 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:38.261440 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:38.261417 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:38.385060 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:38.384189 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7jfqn" Sep 19 05:41:38.385060 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:38.384806 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7jfqn" Sep 19 05:41:38.478891 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:38.478850 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:38.710098 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:38.710006 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:38.950080 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:38.950054 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5fm2w_c30c631d-63ed-410c-8d71-a0f4384bae54/node-ca/0.log" Sep 19 05:41:39.260807 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:39.260772 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:39.260974 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:39.260919 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:39.311033 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:39.311006 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:39.388646 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:39.388625 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 05:41:39.409811 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:39.409609 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:39.538374 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:39.538069 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:39.635188 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:39.635138 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:40.263884 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:40.263854 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:40.264047 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:40.263896 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:40.264047 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:40.263978 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:40.264129 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:40.264058 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:40.396714 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:40.396683 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 05:41:40.397205 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:40.397179 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" event={"ID":"795198a1-531e-4263-89f7-59d4a963ef34","Type":"ContainerStarted","Data":"99dda15b14fb79e8bd7194a3edbbf675b4459b5dd4299a66f6533a376cb659c8"} Sep 19 05:41:40.397481 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:40.397448 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:40.397676 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:40.397657 2584 scope.go:117] "RemoveContainer" containerID="7a17df0dc9718e79c25cca4fddc23b2e5ebd15f0ab7042f3946514a8d3eaeb19" Sep 19 05:41:40.397761 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:40.397682 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:40.397761 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:40.397758 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:40.415003 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:40.414978 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:40.418886 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:40.418854 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:41:40.510971 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:40.510936 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:41.127761 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.127720 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:41.260485 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.260461 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:41.260633 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:41.260608 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:41.285659 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.285556 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:41.402003 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.401965 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gss8z" event={"ID":"653a14cf-5131-4a36-8c13-35f8ab9d499a","Type":"ContainerStarted","Data":"73e6d87990e7598ddd03162b0e0a7cf62d4f527ab5159ffd787e8f350ddf4104"} Sep 19 05:41:41.406232 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.406211 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 05:41:41.406708 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.406656 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" event={"ID":"795198a1-531e-4263-89f7-59d4a963ef34","Type":"ContainerStarted","Data":"cf6f4e2065b91cb5bc3a56a88e2259ef2265059f6d389bd22f0a2e4e70a7cfed"} Sep 19 05:41:41.447927 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.447897 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:41.498581 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.498515 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" podStartSLOduration=9.544617651 podStartE2EDuration="27.498490193s" totalImagesPullingTime="17.953872542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:41.498080766 +0000 UTC m=+29.491308296" watchObservedRunningTime="2026-09-19 05:41:41.498490193 +0000 UTC m=+29.491717724" Sep 19 05:41:41.555167 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.555100 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:41.579009 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.578977 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s9kmn"] Sep 19 05:41:41.579177 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.579104 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:41.579240 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:41.579200 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:41.582373 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.582350 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pnp4s"] Sep 19 05:41:41.582503 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.582444 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:41.582547 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:41.582527 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:41.597216 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.597187 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8947n"] Sep 19 05:41:41.597338 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.597307 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:41.597459 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:41.597438 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:41.812634 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:41.812550 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:42.411158 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:42.411127 2584 generic.go:356] "Generic (PLEG): container finished" podID="653a14cf-5131-4a36-8c13-35f8ab9d499a" containerID="73e6d87990e7598ddd03162b0e0a7cf62d4f527ab5159ffd787e8f350ddf4104" exitCode=0 Sep 19 05:41:42.411559 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:42.411216 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gss8z" event={"ID":"653a14cf-5131-4a36-8c13-35f8ab9d499a","Type":"ContainerDied","Data":"73e6d87990e7598ddd03162b0e0a7cf62d4f527ab5159ffd787e8f350ddf4104"} Sep 19 05:41:42.599021 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:42.598988 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:43.188769 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:43.188734 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:43.260992 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:43.260957 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:43.261175 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:43.260999 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:43.261175 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:43.261048 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:43.261175 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:43.261061 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:43.261175 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:43.261136 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:43.261324 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:43.261189 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:43.392818 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:43.392791 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:43.779749 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:43.779512 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:43.880650 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:43.880614 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:43.999381 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:43.999353 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:44.393058 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:44.392978 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:44.417851 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:44.417823 2584 generic.go:356] "Generic (PLEG): container finished" podID="653a14cf-5131-4a36-8c13-35f8ab9d499a" containerID="f28ac6a6139092f41da78bd8ca09618a0a92c5da52a99c04682cc6735c607b3e" exitCode=0 Sep 19 05:41:44.417977 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:44.417856 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gss8z" event={"ID":"653a14cf-5131-4a36-8c13-35f8ab9d499a","Type":"ContainerDied","Data":"f28ac6a6139092f41da78bd8ca09618a0a92c5da52a99c04682cc6735c607b3e"} Sep 19 05:41:44.783588 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:44.783507 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:45.017168 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:45.017137 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:45.219669 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:45.219631 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:45.260900 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:45.260866 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:45.261054 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:45.260879 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:45.261054 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:45.260885 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:45.261177 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:45.261008 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:45.261177 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:45.261078 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:45.261177 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:45.261129 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:45.387811 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:45.387767 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:45.789455 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:45.789424 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:45.847958 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:45.847927 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:45.989795 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:45.989709 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:46.192702 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:46.192677 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:46.425095 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:46.425062 2584 generic.go:356] "Generic (PLEG): container finished" podID="653a14cf-5131-4a36-8c13-35f8ab9d499a" containerID="ec0d21bcf65123c0e21b42fbf083fa213bf3c23352d2eabdc10702dcd30e1d01" exitCode=0 Sep 19 05:41:46.425095 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:46.425098 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gss8z" event={"ID":"653a14cf-5131-4a36-8c13-35f8ab9d499a","Type":"ContainerDied","Data":"ec0d21bcf65123c0e21b42fbf083fa213bf3c23352d2eabdc10702dcd30e1d01"} Sep 19 05:41:46.851615 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:46.851581 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:46.995328 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:46.995248 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:47.108709 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:47.108623 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:47.260830 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:47.260799 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:47.261035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:47.260904 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:47.261035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:47.260915 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:47.261035 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:47.260915 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:47.261035 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:47.261022 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:47.261215 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:47.261105 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:47.781826 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:47.781724 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:47.894804 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:47.894771 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:47.895205 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:47.894900 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:47.895205 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:47.894959 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret podName:8d49d458-d0eb-4619-9b41-2aa12b664541 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:19.894941383 +0000 UTC m=+67.888168893 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret") pod "global-pull-secret-syncer-s9kmn" (UID: "8d49d458-d0eb-4619-9b41-2aa12b664541") : object "kube-system"/"original-pull-secret" not registered Sep 19 05:41:47.995430 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:47.995387 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:47.995631 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:47.995522 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:47.995631 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:47.995580 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs podName:ff709368-cadb-49c8-8bf1-eb0c3938922c nodeName:}" failed. No retries permitted until 2026-09-19 05:42:19.995563104 +0000 UTC m=+67.988790621 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs") pod "network-metrics-daemon-8947n" (UID: "ff709368-cadb-49c8-8bf1-eb0c3938922c") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 05:41:48.095933 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:48.095903 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288vl\" (UniqueName: \"kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl\") pod \"network-check-target-pnp4s\" (UID: \"27fb3423-d426-4d0c-b117-439667e2be79\") " pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:48.096136 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:48.096088 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 05:41:48.096136 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:48.096111 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 05:41:48.096136 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:48.096122 2584 projected.go:196] Error preparing data for projected volume kube-api-access-288vl for pod openshift-network-diagnostics/network-check-target-pnp4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:48.096301 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:48.096188 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl podName:27fb3423-d426-4d0c-b117-439667e2be79 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:20.096168503 +0000 UTC m=+68.089396035 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-288vl" (UniqueName: "kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl") pod "network-check-target-pnp4s" (UID: "27fb3423-d426-4d0c-b117-439667e2be79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 05:41:48.183294 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:48.183257 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:48.596054 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:48.596000 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:48.991684 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:48.991590 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:49.193939 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:49.193899 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:49.261315 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:49.261226 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:49.261315 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:49.261274 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:49.261315 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:49.261276 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:49.261591 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:49.261356 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:49.261591 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:49.261524 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:49.261698 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:49.261616 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:49.800126 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:49.800093 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:49.900742 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:49.900701 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:50.250825 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:50.250788 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:50.587134 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:50.587098 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:50.787708 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:50.787668 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:51.184919 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:51.184842 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:51.260915 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:51.260876 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:51.260915 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:51.260908 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:51.261320 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:51.260908 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:51.261320 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:51.260983 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:51.261320 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:51.261064 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:51.261320 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:51.261126 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:51.439914 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:51.439833 2584 generic.go:356] "Generic (PLEG): container finished" podID="653a14cf-5131-4a36-8c13-35f8ab9d499a" containerID="d8bd78036fe182a7f6a0a83f81d4523bb1265466f160b384d9290fb28d994dff" exitCode=0 Sep 19 05:41:51.439914 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:51.439881 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gss8z" event={"ID":"653a14cf-5131-4a36-8c13-35f8ab9d499a","Type":"ContainerDied","Data":"d8bd78036fe182a7f6a0a83f81d4523bb1265466f160b384d9290fb28d994dff"} Sep 19 05:41:51.586152 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:51.586124 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:51.786788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:51.786703 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:52.094690 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:52.094665 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:52.191279 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:52.191243 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:52.444769 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:52.444670 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:52.445270 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:52.445247 2584 generic.go:356] "Generic (PLEG): container finished" podID="653a14cf-5131-4a36-8c13-35f8ab9d499a" containerID="fb315c6743e5800a1c91eab47960636733c7850ec5143f8f490357b664246e33" exitCode=0 Sep 19 05:41:52.445364 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:52.445284 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gss8z" event={"ID":"653a14cf-5131-4a36-8c13-35f8ab9d499a","Type":"ContainerDied","Data":"fb315c6743e5800a1c91eab47960636733c7850ec5143f8f490357b664246e33"} Sep 19 05:41:53.261258 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:53.261223 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:53.261452 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:53.261223 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:53.261452 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:53.261328 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:53.261452 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:53.261345 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:53.261637 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:53.261452 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:53.261637 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:53.261521 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:53.450794 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:53.450764 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gss8z" event={"ID":"653a14cf-5131-4a36-8c13-35f8ab9d499a","Type":"ContainerStarted","Data":"a6a230a0079dc2de6ef55ccd775cee36b159253a8e3715695a8d6ee633a26ae8"} Sep 19 05:41:53.474050 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:53.474003 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gss8z" podStartSLOduration=9.391330509 podStartE2EDuration="39.473986806s" totalImagesPullingTime="30.082656297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:41:53.472701568 +0000 UTC m=+41.465929098" watchObservedRunningTime="2026-09-19 05:41:53.473986806 +0000 UTC m=+41.467214337" Sep 19 05:41:53.819795 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:53.819764 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:53.910647 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:53.910618 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:54.082476 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:54.082448 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:54.259300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:54.259267 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:54.373293 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:54.373201 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:54.642877 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:54.642793 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:54.743821 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:54.743779 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:54.849075 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:54.849045 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:54.915911 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:54.915831 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:55.260807 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:55.260704 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:55.260807 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:55.260747 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:55.260807 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:55.260705 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:55.261082 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:55.260834 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:55.261082 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:55.260892 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:55.261082 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:55.260950 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:55.468314 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:55.468287 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:55.796296 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:55.796268 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:56.110258 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:56.110231 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:56.642159 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:56.642121 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:56.751175 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:56.751147 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:56.951132 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:56.951050 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:57.051939 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:57.051901 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:41:57.261105 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:57.261030 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:57.261247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:57.261030 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:57.261247 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:57.261139 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:41:57.261247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:57.261030 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:57.261247 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:57.261211 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:57.261375 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:57.261288 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:59.260650 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:59.260606 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:41:59.260650 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:59.260644 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:41:59.260650 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:41:59.260654 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:41:59.261120 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:59.260717 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:41:59.261120 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:59.260825 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:41:59.261120 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:41:59.260886 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:42:01.261173 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:01.261136 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:42:01.261581 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:01.261235 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:42:01.261581 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:01.261240 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnp4s" podUID="27fb3423-d426-4d0c-b117-439667e2be79" Sep 19 05:42:01.261581 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:01.261264 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:42:01.261581 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:01.261344 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s9kmn" podUID="8d49d458-d0eb-4619-9b41-2aa12b664541" Sep 19 05:42:01.261581 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:01.261416 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8947n" podUID="ff709368-cadb-49c8-8bf1-eb0c3938922c" Sep 19 05:42:01.560271 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:01.560183 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:01.653221 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:01.653185 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:01.754043 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:01.754015 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:01.865199 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:01.865164 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:02.299208 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.299134 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-57.ec2.internal" event="NodeReady" Sep 19 05:42:02.299617 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.299257 2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 19 05:42:02.354336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.354304 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ct6zn"] Sep 19 05:42:02.379482 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.379450 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nz8gq"] Sep 19 05:42:02.380154 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.380129 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ct6zn" Sep 19 05:42:02.382824 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.382801 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 19 05:42:02.383023 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.383003 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 19 05:42:02.383086 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.383010 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 19 05:42:02.383086 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.383059 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q45g7\"" Sep 19 05:42:02.396762 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.396731 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sv6p\" (UniqueName: \"kubernetes.io/projected/4b0a35bb-9988-4069-8c94-38de7dff23cb-kube-api-access-5sv6p\") pod \"ingress-canary-ct6zn\" (UID: \"4b0a35bb-9988-4069-8c94-38de7dff23cb\") " pod="openshift-ingress-canary/ingress-canary-ct6zn" Sep 19 05:42:02.396886 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.396777 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b0a35bb-9988-4069-8c94-38de7dff23cb-cert\") pod \"ingress-canary-ct6zn\" (UID: \"4b0a35bb-9988-4069-8c94-38de7dff23cb\") " pod="openshift-ingress-canary/ingress-canary-ct6zn" Sep 19 05:42:02.397439 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.397420 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ct6zn"] Sep 19 05:42:02.397532 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.397451 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nz8gq"] Sep 19 05:42:02.397532 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.397499 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.400179 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.400160 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 19 05:42:02.400730 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.400703 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s598g\"" Sep 19 05:42:02.400730 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.400704 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 19 05:42:02.400869 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.400735 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 19 05:42:02.400869 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.400705 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 19 05:42:02.454935 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.454900 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lswnv"] Sep 19 05:42:02.467561 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.467530 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.469813 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.469762 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lswnv"] Sep 19 05:42:02.470948 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.470910 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mbwrd\"" Sep 19 05:42:02.471086 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.470982 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 19 05:42:02.471086 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.471044 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 19 05:42:02.497856 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.497818 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe317156-45ab-43ef-b710-c09ac2f3debe-config-volume\") pod \"dns-default-lswnv\" (UID: \"fe317156-45ab-43ef-b710-c09ac2f3debe\") " pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.497856 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.497859 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-data-volume\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.498060 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.497892 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.498060 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.497916 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65p4g\" (UniqueName: \"kubernetes.io/projected/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-kube-api-access-65p4g\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.498060 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.497942 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe317156-45ab-43ef-b710-c09ac2f3debe-metrics-tls\") pod \"dns-default-lswnv\" (UID: \"fe317156-45ab-43ef-b710-c09ac2f3debe\") " pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.498060 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.497980 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-crio-socket\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.498060 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.497998 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe317156-45ab-43ef-b710-c09ac2f3debe-tmp-dir\") pod \"dns-default-lswnv\" (UID: \"fe317156-45ab-43ef-b710-c09ac2f3debe\") " pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.498060 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.498013 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s67fn\" (UniqueName: \"kubernetes.io/projected/fe317156-45ab-43ef-b710-c09ac2f3debe-kube-api-access-s67fn\") pod \"dns-default-lswnv\" (UID: \"fe317156-45ab-43ef-b710-c09ac2f3debe\") " pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.498238 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.498070 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sv6p\" (UniqueName: \"kubernetes.io/projected/4b0a35bb-9988-4069-8c94-38de7dff23cb-kube-api-access-5sv6p\") pod \"ingress-canary-ct6zn\" (UID: \"4b0a35bb-9988-4069-8c94-38de7dff23cb\") " pod="openshift-ingress-canary/ingress-canary-ct6zn" Sep 19 05:42:02.498238 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.498104 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.498238 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.498127 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b0a35bb-9988-4069-8c94-38de7dff23cb-cert\") pod \"ingress-canary-ct6zn\" (UID: \"4b0a35bb-9988-4069-8c94-38de7dff23cb\") " pod="openshift-ingress-canary/ingress-canary-ct6zn" Sep 19 05:42:02.502311 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.502278 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b0a35bb-9988-4069-8c94-38de7dff23cb-cert\") pod \"ingress-canary-ct6zn\" (UID: \"4b0a35bb-9988-4069-8c94-38de7dff23cb\") " pod="openshift-ingress-canary/ingress-canary-ct6zn" Sep 19 05:42:02.506413 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.506370 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sv6p\" (UniqueName: \"kubernetes.io/projected/4b0a35bb-9988-4069-8c94-38de7dff23cb-kube-api-access-5sv6p\") pod \"ingress-canary-ct6zn\" (UID: \"4b0a35bb-9988-4069-8c94-38de7dff23cb\") " pod="openshift-ingress-canary/ingress-canary-ct6zn" Sep 19 05:42:02.581198 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.581166 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:02.598440 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.598387 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-data-volume\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.598528 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.598457 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.598528 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.598478 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65p4g\" (UniqueName: \"kubernetes.io/projected/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-kube-api-access-65p4g\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.598528 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.598502 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe317156-45ab-43ef-b710-c09ac2f3debe-metrics-tls\") pod \"dns-default-lswnv\" (UID: \"fe317156-45ab-43ef-b710-c09ac2f3debe\") " pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.598528 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.598521 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-crio-socket\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.598713 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.598536 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe317156-45ab-43ef-b710-c09ac2f3debe-tmp-dir\") pod \"dns-default-lswnv\" (UID: \"fe317156-45ab-43ef-b710-c09ac2f3debe\") " pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.598713 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.598557 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s67fn\" (UniqueName: \"kubernetes.io/projected/fe317156-45ab-43ef-b710-c09ac2f3debe-kube-api-access-s67fn\") pod \"dns-default-lswnv\" (UID: \"fe317156-45ab-43ef-b710-c09ac2f3debe\") " pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.598713 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.598611 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.598713 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.598662 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe317156-45ab-43ef-b710-c09ac2f3debe-config-volume\") pod \"dns-default-lswnv\" (UID: \"fe317156-45ab-43ef-b710-c09ac2f3debe\") " pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.598881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.598761 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-crio-socket\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.598881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.598811 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-data-volume\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.598971 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.598898 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe317156-45ab-43ef-b710-c09ac2f3debe-tmp-dir\") pod \"dns-default-lswnv\" (UID: \"fe317156-45ab-43ef-b710-c09ac2f3debe\") " pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.599192 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.599166 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.599253 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.599205 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fe317156-45ab-43ef-b710-c09ac2f3debe-config-volume\") pod \"dns-default-lswnv\" (UID: \"fe317156-45ab-43ef-b710-c09ac2f3debe\") " pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.600969 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.600951 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.601043 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.601014 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe317156-45ab-43ef-b710-c09ac2f3debe-metrics-tls\") pod \"dns-default-lswnv\" (UID: \"fe317156-45ab-43ef-b710-c09ac2f3debe\") " pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.607633 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.607608 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s67fn\" (UniqueName: \"kubernetes.io/projected/fe317156-45ab-43ef-b710-c09ac2f3debe-kube-api-access-s67fn\") pod \"dns-default-lswnv\" (UID: \"fe317156-45ab-43ef-b710-c09ac2f3debe\") " pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:02.608917 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.608900 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65p4g\" (UniqueName: \"kubernetes.io/projected/3acd80a4-9d7b-4f80-98e7-68d5429e8bc3-kube-api-access-65p4g\") pod \"insights-runtime-extractor-nz8gq\" (UID: \"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3\") " pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.685372 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.685344 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:02.689638 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.689623 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ct6zn" Sep 19 05:42:02.705994 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.705967 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nz8gq" Sep 19 05:42:02.777786 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:02.777752 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:03.042393 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.042362 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ct6zn"] Sep 19 05:42:03.045321 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.045293 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nz8gq"] Sep 19 05:42:03.046189 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:03.046163 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0a35bb_9988_4069_8c94_38de7dff23cb.slice/crio-f2721421965d1b190b4c5ebb046aa4c79eb53491cd20bd167255d40efc685a7d WatchSource:0}: Error finding container f2721421965d1b190b4c5ebb046aa4c79eb53491cd20bd167255d40efc685a7d: Status 404 returned error can't find the container with id f2721421965d1b190b4c5ebb046aa4c79eb53491cd20bd167255d40efc685a7d Sep 19 05:42:03.048331 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:03.048312 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3acd80a4_9d7b_4f80_98e7_68d5429e8bc3.slice/crio-b97529654f85889a3f2db0e8ed145ea923b696f2d795de879ddea8c69f5687c3 WatchSource:0}: Error finding container b97529654f85889a3f2db0e8ed145ea923b696f2d795de879ddea8c69f5687c3: Status 404 returned error can't find the container with id b97529654f85889a3f2db0e8ed145ea923b696f2d795de879ddea8c69f5687c3 Sep 19 05:42:03.087355 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.087325 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lswnv"] Sep 19 05:42:03.105565 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:03.105529 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe317156_45ab_43ef_b710_c09ac2f3debe.slice/crio-7f3852b9e7bd10491c962aa2e578c74ae4b88099d4b4546c0f3eadaa66e672ff WatchSource:0}: Error finding container 7f3852b9e7bd10491c962aa2e578c74ae4b88099d4b4546c0f3eadaa66e672ff: Status 404 returned error can't find the container with id 7f3852b9e7bd10491c962aa2e578c74ae4b88099d4b4546c0f3eadaa66e672ff Sep 19 05:42:03.261181 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.261139 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:42:03.261181 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.261180 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:42:03.261428 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.261139 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:42:03.263830 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.263807 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 19 05:42:03.264045 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.263893 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 19 05:42:03.264045 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.263913 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tjt75\"" Sep 19 05:42:03.264045 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.263945 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 19 05:42:03.264743 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.264724 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 19 05:42:03.264844 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.264761 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b2b8g\"" Sep 19 05:42:03.468316 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.468231 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fc5999b4b-nrk72"] Sep 19 05:42:03.490517 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.490473 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lswnv" event={"ID":"fe317156-45ab-43ef-b710-c09ac2f3debe","Type":"ContainerStarted","Data":"7f3852b9e7bd10491c962aa2e578c74ae4b88099d4b4546c0f3eadaa66e672ff"} Sep 19 05:42:03.490517 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.490515 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nz8gq" event={"ID":"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3","Type":"ContainerStarted","Data":"d4727c30ee865ffce5a235c90c245db9532983cccd2308b7d9adff4b0a4c7cb6"} Sep 19 05:42:03.490710 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.490550 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nz8gq" event={"ID":"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3","Type":"ContainerStarted","Data":"b97529654f85889a3f2db0e8ed145ea923b696f2d795de879ddea8c69f5687c3"} Sep 19 05:42:03.490710 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.490570 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ct6zn" event={"ID":"4b0a35bb-9988-4069-8c94-38de7dff23cb","Type":"ContainerStarted","Data":"f2721421965d1b190b4c5ebb046aa4c79eb53491cd20bd167255d40efc685a7d"} Sep 19 05:42:03.490710 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.490590 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fc5999b4b-nrk72"] Sep 19 05:42:03.490710 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.490680 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.493154 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.493122 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 19 05:42:03.493684 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.493640 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 19 05:42:03.493684 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.493654 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 19 05:42:03.493880 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.493203 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tlm5t\"" Sep 19 05:42:03.493880 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.493728 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 19 05:42:03.494628 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.494110 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 19 05:42:03.494628 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.494498 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 19 05:42:03.494810 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.494705 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 19 05:42:03.504384 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.504359 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9774\" (UniqueName: \"kubernetes.io/projected/9300805f-f719-422b-a428-ae038d9e6a2c-kube-api-access-p9774\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.504539 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.504392 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9300805f-f719-422b-a428-ae038d9e6a2c-console-serving-cert\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.504539 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.504474 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9300805f-f719-422b-a428-ae038d9e6a2c-console-oauth-config\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.504539 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.504501 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-service-ca\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.504539 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.504527 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-oauth-serving-cert\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.504662 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.504546 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-console-config\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.605456 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.605410 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-service-ca\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.605649 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.605466 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-oauth-serving-cert\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.605649 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.605488 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-console-config\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.605649 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.605554 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9774\" (UniqueName: \"kubernetes.io/projected/9300805f-f719-422b-a428-ae038d9e6a2c-kube-api-access-p9774\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.605649 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.605578 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9300805f-f719-422b-a428-ae038d9e6a2c-console-serving-cert\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.605649 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.605628 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9300805f-f719-422b-a428-ae038d9e6a2c-console-oauth-config\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.606275 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.606245 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-service-ca\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.606675 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.606646 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-console-config\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.607222 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.607180 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-oauth-serving-cert\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.608798 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.608763 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9300805f-f719-422b-a428-ae038d9e6a2c-console-oauth-config\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.609300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.609276 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9300805f-f719-422b-a428-ae038d9e6a2c-console-serving-cert\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.615875 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.615468 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9774\" (UniqueName: \"kubernetes.io/projected/9300805f-f719-422b-a428-ae038d9e6a2c-kube-api-access-p9774\") pod \"console-fc5999b4b-nrk72\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.801271 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.801177 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:03.891253 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.891178 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:03.959391 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:03.959352 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:04.245262 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:04.245232 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fc5999b4b-nrk72"] Sep 19 05:42:04.292914 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:04.292870 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9300805f_f719_422b_a428_ae038d9e6a2c.slice/crio-d9942b1f218a5c20f780049f9008a8d2036952656f497e3a221d06acf9732166 WatchSource:0}: Error finding container d9942b1f218a5c20f780049f9008a8d2036952656f497e3a221d06acf9732166: Status 404 returned error can't find the container with id d9942b1f218a5c20f780049f9008a8d2036952656f497e3a221d06acf9732166 Sep 19 05:42:04.483480 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:04.483440 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nz8gq" event={"ID":"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3","Type":"ContainerStarted","Data":"f0e5ddce319e9d7fd90d812aa85cf384224fe2f7826e1c06b60d5fbe7b234eec"} Sep 19 05:42:04.485623 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:04.485587 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc5999b4b-nrk72" event={"ID":"9300805f-f719-422b-a428-ae038d9e6a2c","Type":"ContainerStarted","Data":"d9942b1f218a5c20f780049f9008a8d2036952656f497e3a221d06acf9732166"} Sep 19 05:42:05.210607 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:05.210567 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:05.313371 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:05.313301 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:06.092030 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:06.091985 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:06.160227 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:06.160137 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:06.500820 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:06.500710 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ct6zn" event={"ID":"4b0a35bb-9988-4069-8c94-38de7dff23cb","Type":"ContainerStarted","Data":"3c7b946b59a2b7a777edb96609c36176a3f2e43242c9650b4eeb171b5a22f613"} Sep 19 05:42:06.505128 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:06.505101 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lswnv" event={"ID":"fe317156-45ab-43ef-b710-c09ac2f3debe","Type":"ContainerStarted","Data":"8c29d6eebb38e9d55b69725f5272f32faee74684f45e2905cecbc768156fe9f2"} Sep 19 05:42:06.505378 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:06.505340 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lswnv" event={"ID":"fe317156-45ab-43ef-b710-c09ac2f3debe","Type":"ContainerStarted","Data":"25bc277e65681fbb256f503e5fcb0f61377a5ec966a404820f700c14a6eec3eb"} Sep 19 05:42:06.505566 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:06.505521 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:06.545116 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:06.544887 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ct6zn" podStartSLOduration=1.4248014150000001 podStartE2EDuration="4.544863156s" totalImagesPullingTime="3.120061741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:06.522045129 +0000 UTC m=+54.515272661" watchObservedRunningTime="2026-09-19 05:42:06.544863156 +0000 UTC m=+54.538090688" Sep 19 05:42:06.545116 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:06.545011 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lswnv" podStartSLOduration=1.4948226039999999 podStartE2EDuration="4.545004744s" totalImagesPullingTime="3.05018214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:06.543752213 +0000 UTC m=+54.536979745" watchObservedRunningTime="2026-09-19 05:42:06.545004744 +0000 UTC m=+54.538232274" Sep 19 05:42:07.475075 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:07.475043 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:07.583516 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:07.583478 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:07.641905 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:07.641862 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8"] Sep 19 05:42:07.660839 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:07.660804 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8"] Sep 19 05:42:07.661024 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:07.660948 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8" Sep 19 05:42:07.663494 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:07.663471 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 19 05:42:07.664068 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:07.664048 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-sdsrg\"" Sep 19 05:42:07.683055 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:07.683008 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:07.734271 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:07.734180 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/25b751f3-4663-4f33-9170-ebeb996374c8-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-5krs8\" (UID: \"25b751f3-4663-4f33-9170-ebeb996374c8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8" Sep 19 05:42:07.835333 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:07.835288 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/25b751f3-4663-4f33-9170-ebeb996374c8-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-5krs8\" (UID: \"25b751f3-4663-4f33-9170-ebeb996374c8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8" Sep 19 05:42:07.835538 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:07.835455 2584 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 19 05:42:07.835598 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:07.835539 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25b751f3-4663-4f33-9170-ebeb996374c8-tls-certificates podName:25b751f3-4663-4f33-9170-ebeb996374c8 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:08.335517176 +0000 UTC m=+56.328744694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/25b751f3-4663-4f33-9170-ebeb996374c8-tls-certificates") pod "prometheus-operator-admission-webhook-5c6747dc7b-5krs8" (UID: "25b751f3-4663-4f33-9170-ebeb996374c8") : secret "prometheus-operator-admission-webhook-tls" not found Sep 19 05:42:07.906926 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:07.906892 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:08.339262 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:08.339230 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/25b751f3-4663-4f33-9170-ebeb996374c8-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-5krs8\" (UID: \"25b751f3-4663-4f33-9170-ebeb996374c8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8" Sep 19 05:42:08.341991 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:08.341967 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/25b751f3-4663-4f33-9170-ebeb996374c8-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-5krs8\" (UID: \"25b751f3-4663-4f33-9170-ebeb996374c8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8" Sep 19 05:42:08.571388 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:08.571298 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8" Sep 19 05:42:09.095734 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:09.095695 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:09.160936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:09.160901 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:10.174135 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:10.174059 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8"] Sep 19 05:42:10.177266 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:10.177234 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b751f3_4663_4f33_9170_ebeb996374c8.slice/crio-685f1ce3598ee9c12ca5b063478e30eacb9728e30d572c0816021e50e0fba6d7 WatchSource:0}: Error finding container 685f1ce3598ee9c12ca5b063478e30eacb9728e30d572c0816021e50e0fba6d7: Status 404 returned error can't find the container with id 685f1ce3598ee9c12ca5b063478e30eacb9728e30d572c0816021e50e0fba6d7 Sep 19 05:42:10.480824 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:10.480747 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:10.520509 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:10.520469 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8" event={"ID":"25b751f3-4663-4f33-9170-ebeb996374c8","Type":"ContainerStarted","Data":"685f1ce3598ee9c12ca5b063478e30eacb9728e30d572c0816021e50e0fba6d7"} Sep 19 05:42:10.522563 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:10.522537 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nz8gq" event={"ID":"3acd80a4-9d7b-4f80-98e7-68d5429e8bc3","Type":"ContainerStarted","Data":"d058c45b7ae0baa8547d9749d4bbf3bf7d139f47c63280f3780b8788bf10fc78"} Sep 19 05:42:10.523963 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:10.523942 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc5999b4b-nrk72" event={"ID":"9300805f-f719-422b-a428-ae038d9e6a2c","Type":"ContainerStarted","Data":"e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192"} Sep 19 05:42:10.553438 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:10.553374 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nz8gq" podStartSLOduration=1.952906662 podStartE2EDuration="8.553360178s" totalImagesPullingTime="6.600453516s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:42:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:10.552426933 +0000 UTC m=+58.545654469" watchObservedRunningTime="2026-09-19 05:42:10.553360178 +0000 UTC m=+58.546587708" Sep 19 05:42:10.574227 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:10.573991 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fc5999b4b-nrk72" podStartSLOduration=1.93986294 podStartE2EDuration="7.573976101s" totalImagesPullingTime="5.634113161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:10.573573335 +0000 UTC m=+58.566800866" watchObservedRunningTime="2026-09-19 05:42:10.573976101 +0000 UTC m=+58.567203631" Sep 19 05:42:10.586500 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:10.586473 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:11.142688 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.142648 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:11.206736 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.206686 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:11.531585 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.531482 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8" event={"ID":"25b751f3-4663-4f33-9170-ebeb996374c8","Type":"ContainerStarted","Data":"c55106bd1bb2d3714a1b5e96f2688794c360505be8012d7e6b6e3aa180b94837"} Sep 19 05:42:11.549519 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.549476 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8" podStartSLOduration=3.391083487 podStartE2EDuration="4.549460167s" totalImagesPullingTime="1.15837668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:11.549276711 +0000 UTC m=+59.542504242" watchObservedRunningTime="2026-09-19 05:42:11.549460167 +0000 UTC m=+59.542687698" Sep 19 05:42:11.878955 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.878925 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66d6f87bd9-zxf26"] Sep 19 05:42:11.881900 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.881884 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:11.889109 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.889083 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 19 05:42:11.893782 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.893750 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66d6f87bd9-zxf26"] Sep 19 05:42:11.964659 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.964613 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-oauth-serving-cert\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:11.964659 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.964652 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8hb9\" (UniqueName: \"kubernetes.io/projected/b471a970-9591-4089-92bb-ca69f4d48c91-kube-api-access-p8hb9\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:11.964881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.964684 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b471a970-9591-4089-92bb-ca69f4d48c91-console-serving-cert\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:11.964881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.964715 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-trusted-ca-bundle\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:11.964881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.964753 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-console-config\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:11.964881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.964770 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-service-ca\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:11.964881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:11.964824 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b471a970-9591-4089-92bb-ca69f4d48c91-console-oauth-config\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.065663 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.065614 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b471a970-9591-4089-92bb-ca69f4d48c91-console-serving-cert\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.065768 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.065678 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-trusted-ca-bundle\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.065870 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.065849 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-console-config\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.065917 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.065892 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-service-ca\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.066036 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.066015 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b471a970-9591-4089-92bb-ca69f4d48c91-console-oauth-config\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.066152 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.066128 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-oauth-serving-cert\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.066213 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.066165 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8hb9\" (UniqueName: \"kubernetes.io/projected/b471a970-9591-4089-92bb-ca69f4d48c91-kube-api-access-p8hb9\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.066564 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.066538 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-service-ca\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.066564 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.066551 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-console-config\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.066692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.066592 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-trusted-ca-bundle\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.066762 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.066744 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-oauth-serving-cert\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.068311 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.068286 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b471a970-9591-4089-92bb-ca69f4d48c91-console-serving-cert\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.068419 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.068362 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b471a970-9591-4089-92bb-ca69f4d48c91-console-oauth-config\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.074737 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.074706 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8hb9\" (UniqueName: \"kubernetes.io/projected/b471a970-9591-4089-92bb-ca69f4d48c91-kube-api-access-p8hb9\") pod \"console-66d6f87bd9-zxf26\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.190791 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.190696 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:12.422485 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.422455 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzjr9" Sep 19 05:42:12.463344 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.463263 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66d6f87bd9-zxf26"] Sep 19 05:42:12.466509 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:12.466476 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb471a970_9591_4089_92bb_ca69f4d48c91.slice/crio-c91ae7465e77625a3ad5d955ad37786380f0adafeaa0e8bf84bea47f227141dd WatchSource:0}: Error finding container c91ae7465e77625a3ad5d955ad37786380f0adafeaa0e8bf84bea47f227141dd: Status 404 returned error can't find the container with id c91ae7465e77625a3ad5d955ad37786380f0adafeaa0e8bf84bea47f227141dd Sep 19 05:42:12.536537 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.536508 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d6f87bd9-zxf26" event={"ID":"b471a970-9591-4089-92bb-ca69f4d48c91","Type":"ContainerStarted","Data":"c91ae7465e77625a3ad5d955ad37786380f0adafeaa0e8bf84bea47f227141dd"} Sep 19 05:42:12.536679 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.536661 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8" Sep 19 05:42:12.541591 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.541568 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-5krs8" Sep 19 05:42:12.667452 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:12.667421 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:13.001017 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.000983 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:13.129100 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.129068 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:13.225348 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.225318 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:13.295387 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.295315 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:13.541614 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.541567 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d6f87bd9-zxf26" event={"ID":"b471a970-9591-4089-92bb-ca69f4d48c91","Type":"ContainerStarted","Data":"b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917"} Sep 19 05:42:13.565422 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.565299 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66d6f87bd9-zxf26" podStartSLOduration=2.56528016 podStartE2EDuration="2.56528016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:13.563898146 +0000 UTC m=+61.557125676" watchObservedRunningTime="2026-09-19 05:42:13.56528016 +0000 UTC m=+61.558507691" Sep 19 05:42:13.670866 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.670831 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:13.712938 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.712903 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s"] Sep 19 05:42:13.716504 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.716486 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:13.718861 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.718824 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 19 05:42:13.718861 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.718857 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 19 05:42:13.719088 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.719071 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 19 05:42:13.719202 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.719186 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-k7hrv\"" Sep 19 05:42:13.719288 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.719274 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 19 05:42:13.719349 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.719297 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 19 05:42:13.729123 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.729097 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s"] Sep 19 05:42:13.757121 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.757092 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:13.782561 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.782533 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:13.782657 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.782587 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:13.782657 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.782647 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8pzb\" (UniqueName: \"kubernetes.io/projected/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-kube-api-access-f8pzb\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:13.782722 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.782685 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:13.801615 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.801588 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:13.801717 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.801642 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:13.806231 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.806208 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:13.884059 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.884021 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:13.884217 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:13.884158 2584 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 19 05:42:13.884273 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:13.884225 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-prometheus-operator-tls podName:e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f nodeName:}" failed. No retries permitted until 2026-09-19 05:42:14.384205518 +0000 UTC m=+62.377433032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-s7q4s" (UID: "e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f") : secret "prometheus-operator-tls" not found Sep 19 05:42:13.884327 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.884274 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:13.884327 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.884304 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8pzb\" (UniqueName: \"kubernetes.io/projected/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-kube-api-access-f8pzb\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:13.884446 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.884370 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:13.885099 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.885079 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:13.886672 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.886652 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:13.893336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:13.893312 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8pzb\" (UniqueName: \"kubernetes.io/projected/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-kube-api-access-f8pzb\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:14.388012 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:14.387977 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:14.388195 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:14.388094 2584 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 19 05:42:14.388195 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:14.388147 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-prometheus-operator-tls podName:e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f nodeName:}" failed. No retries permitted until 2026-09-19 05:42:15.38813417 +0000 UTC m=+63.381361679 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-s7q4s" (UID: "e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f") : secret "prometheus-operator-tls" not found Sep 19 05:42:14.548233 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:14.548207 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:14.622174 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:14.622142 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:15.395325 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:15.395284 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:15.397680 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:15.397651 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-s7q4s\" (UID: \"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:15.528537 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:15.528506 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-k7hrv\"" Sep 19 05:42:15.536977 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:15.536950 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" Sep 19 05:42:15.830535 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:15.830510 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s"] Sep 19 05:42:15.833245 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:15.833213 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6bc30ff_ee2d_4d45_8d42_db7a6a6e842f.slice/crio-e98b8439f2126cb7638ac63bcad97688585f377fc953e4aa95b6172b3614426a WatchSource:0}: Error finding container e98b8439f2126cb7638ac63bcad97688585f377fc953e4aa95b6172b3614426a: Status 404 returned error can't find the container with id e98b8439f2126cb7638ac63bcad97688585f377fc953e4aa95b6172b3614426a Sep 19 05:42:16.512059 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:16.512027 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lswnv" Sep 19 05:42:16.551441 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:16.551389 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" event={"ID":"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f","Type":"ContainerStarted","Data":"e98b8439f2126cb7638ac63bcad97688585f377fc953e4aa95b6172b3614426a"} Sep 19 05:42:16.684900 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:16.684862 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:16.787963 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:16.787892 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:17.425687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:17.425606 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:17.556334 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:17.556302 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" event={"ID":"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f","Type":"ContainerStarted","Data":"7fd40a75688b95be5649ea8fda3e4122e94e26dd7fc9ad7a470bb76a62b89a8b"} Sep 19 05:42:17.556541 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:17.556351 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" event={"ID":"e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f","Type":"ContainerStarted","Data":"7060f17ad05f2fcf9467ad4bf08a004137e8e39aeafd2113c2c0b9f40d9fa239"} Sep 19 05:42:17.575076 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:17.575029 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-s7q4s" podStartSLOduration=3.277579815 podStartE2EDuration="4.575012027s" totalImagesPullingTime="1.297432212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:17.574231384 +0000 UTC m=+65.567458915" watchObservedRunningTime="2026-09-19 05:42:17.575012027 +0000 UTC m=+65.568239586" Sep 19 05:42:18.492873 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:18.492838 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:18.594639 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:18.594610 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:18.871035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:18.871002 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:18.989832 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:18.989802 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:19.130449 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.130354 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:19.160064 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.160033 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl"] Sep 19 05:42:19.163213 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.163193 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.166162 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.166144 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 19 05:42:19.167326 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.167296 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-s4gwv"] Sep 19 05:42:19.167465 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.167447 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 19 05:42:19.167530 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.167511 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9v8kx\"" Sep 19 05:42:19.170334 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.170317 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.173095 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.173074 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 19 05:42:19.173230 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.173119 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 19 05:42:19.173230 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.173141 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-59bvr\"" Sep 19 05:42:19.177234 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.177216 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 19 05:42:19.180975 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.180954 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl"] Sep 19 05:42:19.202306 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.202280 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-s4gwv"] Sep 19 05:42:19.203275 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.203259 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cjrpq"] Sep 19 05:42:19.206857 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.206842 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.209137 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.209119 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-x6wsd\"" Sep 19 05:42:19.209348 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.209329 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 19 05:42:19.209460 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.209329 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 19 05:42:19.209536 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.209490 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 19 05:42:19.228125 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.228101 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3335bcd0-e181-42c7-a694-2191e06d524c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.228259 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.228145 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07b5c728-7106-4bab-87f1-007e05822927-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.228259 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.228180 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3335bcd0-e181-42c7-a694-2191e06d524c-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.228259 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.228213 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3335bcd0-e181-42c7-a694-2191e06d524c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.228471 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.228281 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/07b5c728-7106-4bab-87f1-007e05822927-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.228471 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.228310 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07b5c728-7106-4bab-87f1-007e05822927-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.228471 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.228337 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/07b5c728-7106-4bab-87f1-007e05822927-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.228471 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.228370 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsdnc\" (UniqueName: \"kubernetes.io/projected/3335bcd0-e181-42c7-a694-2191e06d524c-kube-api-access-fsdnc\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.228471 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.228418 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbkr7\" (UniqueName: \"kubernetes.io/projected/07b5c728-7106-4bab-87f1-007e05822927-kube-api-access-rbkr7\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.228722 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.228493 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07b5c728-7106-4bab-87f1-007e05822927-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.252853 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.252821 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:19.329352 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.329316 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3335bcd0-e181-42c7-a694-2191e06d524c-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.329352 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.329355 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3335bcd0-e181-42c7-a694-2191e06d524c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.329632 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.329392 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/07b5c728-7106-4bab-87f1-007e05822927-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.329632 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.329430 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e87e7c2f-8946-40ef-8198-2daac6a33144-sys\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.329632 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.329548 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07b5c728-7106-4bab-87f1-007e05822927-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.329632 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.329577 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e87e7c2f-8946-40ef-8198-2daac6a33144-root\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.329632 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.329598 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e87e7c2f-8946-40ef-8198-2daac6a33144-metrics-client-ca\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.329632 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.329625 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/07b5c728-7106-4bab-87f1-007e05822927-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.329904 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.329657 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsdnc\" (UniqueName: \"kubernetes.io/projected/3335bcd0-e181-42c7-a694-2191e06d524c-kube-api-access-fsdnc\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.329904 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.329686 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbkr7\" (UniqueName: \"kubernetes.io/projected/07b5c728-7106-4bab-87f1-007e05822927-kube-api-access-rbkr7\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.329904 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.329716 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-tls\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.329904 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.329741 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07b5c728-7106-4bab-87f1-007e05822927-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.330098 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.330032 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-textfile\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.330098 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.330080 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-accelerators-collector-config\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.330203 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.330158 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6tsv\" (UniqueName: \"kubernetes.io/projected/e87e7c2f-8946-40ef-8198-2daac6a33144-kube-api-access-d6tsv\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.330203 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.330195 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.330373 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.330243 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-wtmp\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.330373 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.330260 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/07b5c728-7106-4bab-87f1-007e05822927-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.330373 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.330297 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3335bcd0-e181-42c7-a694-2191e06d524c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.330373 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.330328 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07b5c728-7106-4bab-87f1-007e05822927-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.330592 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:19.330428 2584 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 19 05:42:19.330592 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:19.330509 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3335bcd0-e181-42c7-a694-2191e06d524c-openshift-state-metrics-tls podName:3335bcd0-e181-42c7-a694-2191e06d524c nodeName:}" failed. No retries permitted until 2026-09-19 05:42:19.830488605 +0000 UTC m=+67.823716131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3335bcd0-e181-42c7-a694-2191e06d524c-openshift-state-metrics-tls") pod "openshift-state-metrics-86c5bc79d6-r8tjl" (UID: "3335bcd0-e181-42c7-a694-2191e06d524c") : secret "openshift-state-metrics-tls" not found Sep 19 05:42:19.330734 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.330686 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/07b5c728-7106-4bab-87f1-007e05822927-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.332338 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.332316 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07b5c728-7106-4bab-87f1-007e05822927-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.332690 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.332671 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3335bcd0-e181-42c7-a694-2191e06d524c-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.333507 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.333483 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07b5c728-7106-4bab-87f1-007e05822927-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.334525 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.334503 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07b5c728-7106-4bab-87f1-007e05822927-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.335146 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.335097 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3335bcd0-e181-42c7-a694-2191e06d524c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.339497 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.339469 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsdnc\" (UniqueName: \"kubernetes.io/projected/3335bcd0-e181-42c7-a694-2191e06d524c-kube-api-access-fsdnc\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.340800 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.340783 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbkr7\" (UniqueName: \"kubernetes.io/projected/07b5c728-7106-4bab-87f1-007e05822927-kube-api-access-rbkr7\") pod \"kube-state-metrics-84c885658c-s4gwv\" (UID: \"07b5c728-7106-4bab-87f1-007e05822927\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.346301 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.346280 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:19.431772 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.431683 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e87e7c2f-8946-40ef-8198-2daac6a33144-sys\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.431772 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.431717 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e87e7c2f-8946-40ef-8198-2daac6a33144-root\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.431772 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.431737 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e87e7c2f-8946-40ef-8198-2daac6a33144-metrics-client-ca\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.431772 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.431763 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-tls\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.432091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.431790 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-textfile\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.432091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.431808 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-accelerators-collector-config\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.432091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.431813 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e87e7c2f-8946-40ef-8198-2daac6a33144-root\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.432091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.431827 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6tsv\" (UniqueName: \"kubernetes.io/projected/e87e7c2f-8946-40ef-8198-2daac6a33144-kube-api-access-d6tsv\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.432091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.431888 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e87e7c2f-8946-40ef-8198-2daac6a33144-sys\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.432091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.431938 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.432091 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:19.431953 2584 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 19 05:42:19.432091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.431979 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-wtmp\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.432091 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:19.432025 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-tls podName:e87e7c2f-8946-40ef-8198-2daac6a33144 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:19.932006898 +0000 UTC m=+67.925234406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-tls") pod "node-exporter-cjrpq" (UID: "e87e7c2f-8946-40ef-8198-2daac6a33144") : secret "node-exporter-tls" not found Sep 19 05:42:19.432560 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.432115 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-wtmp\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.432560 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.432282 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-textfile\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.432560 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.432511 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-accelerators-collector-config\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.432560 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.432515 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e87e7c2f-8946-40ef-8198-2daac6a33144-metrics-client-ca\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.434254 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.434222 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.442885 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.442860 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6tsv\" (UniqueName: \"kubernetes.io/projected/e87e7c2f-8946-40ef-8198-2daac6a33144-kube-api-access-d6tsv\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.445985 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.445964 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:19.479251 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.479228 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" Sep 19 05:42:19.546763 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.546736 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:19.649524 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.649490 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:19.750902 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.750778 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:19.751703 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.751682 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-s4gwv"] Sep 19 05:42:19.753575 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:19.753546 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b5c728_7106_4bab_87f1_007e05822927.slice/crio-8a39794ceebd3a404c22d7907caf82c59e7019e8abc7529980cc079f4c26b5cd WatchSource:0}: Error finding container 8a39794ceebd3a404c22d7907caf82c59e7019e8abc7529980cc079f4c26b5cd: Status 404 returned error can't find the container with id 8a39794ceebd3a404c22d7907caf82c59e7019e8abc7529980cc079f4c26b5cd Sep 19 05:42:19.835923 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.835890 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3335bcd0-e181-42c7-a694-2191e06d524c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.838295 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.838273 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3335bcd0-e181-42c7-a694-2191e06d524c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-r8tjl\" (UID: \"3335bcd0-e181-42c7-a694-2191e06d524c\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:19.937283 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.937247 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-tls\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:19.937490 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.937300 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:42:19.937490 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:19.937427 2584 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 19 05:42:19.937584 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:19.937493 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-tls podName:e87e7c2f-8946-40ef-8198-2daac6a33144 nodeName:}" failed. No retries permitted until 2026-09-19 05:42:20.937476819 +0000 UTC m=+68.930704328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-tls") pod "node-exporter-cjrpq" (UID: "e87e7c2f-8946-40ef-8198-2daac6a33144") : secret "node-exporter-tls" not found Sep 19 05:42:19.940044 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.940028 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 19 05:42:19.950022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:19.950004 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d49d458-d0eb-4619-9b41-2aa12b664541-original-pull-secret\") pod \"global-pull-secret-syncer-s9kmn\" (UID: \"8d49d458-d0eb-4619-9b41-2aa12b664541\") " pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:42:20.038155 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.038076 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:42:20.044256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.044238 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 19 05:42:20.050833 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.050815 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff709368-cadb-49c8-8bf1-eb0c3938922c-metrics-certs\") pod \"network-metrics-daemon-8947n\" (UID: \"ff709368-cadb-49c8-8bf1-eb0c3938922c\") " pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:42:20.072134 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.072107 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" Sep 19 05:42:20.073612 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.073596 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tjt75\"" Sep 19 05:42:20.076117 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.076101 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s9kmn" Sep 19 05:42:20.081579 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.081561 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8947n" Sep 19 05:42:20.140194 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.139547 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288vl\" (UniqueName: \"kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl\") pod \"network-check-target-pnp4s\" (UID: \"27fb3423-d426-4d0c-b117-439667e2be79\") " pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:42:20.156355 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.152710 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 19 05:42:20.167276 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.167083 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 19 05:42:20.176759 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.176729 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-288vl\" (UniqueName: \"kubernetes.io/projected/27fb3423-d426-4d0c-b117-439667e2be79-kube-api-access-288vl\") pod \"network-check-target-pnp4s\" (UID: \"27fb3423-d426-4d0c-b117-439667e2be79\") " pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:42:20.219889 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.219760 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:20.356526 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.356487 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:20.371544 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.371424 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s9kmn"] Sep 19 05:42:20.373127 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.373056 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8947n"] Sep 19 05:42:20.374035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.374015 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl"] Sep 19 05:42:20.377000 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:20.376975 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d49d458_d0eb_4619_9b41_2aa12b664541.slice/crio-a35b46ac4877749a11f8e8fdc0addb77ea21db0dbea4209e6c3752374f41c4a5 WatchSource:0}: Error finding container a35b46ac4877749a11f8e8fdc0addb77ea21db0dbea4209e6c3752374f41c4a5: Status 404 returned error can't find the container with id a35b46ac4877749a11f8e8fdc0addb77ea21db0dbea4209e6c3752374f41c4a5 Sep 19 05:42:20.378442 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:20.378394 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff709368_cadb_49c8_8bf1_eb0c3938922c.slice/crio-d11b145a89e93c16907128d34d0a9c1312fe071ca4788397c9361fefa0216fcd WatchSource:0}: Error finding container d11b145a89e93c16907128d34d0a9c1312fe071ca4788397c9361fefa0216fcd: Status 404 returned error can't find the container with id d11b145a89e93c16907128d34d0a9c1312fe071ca4788397c9361fefa0216fcd Sep 19 05:42:20.379188 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:20.379061 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3335bcd0_e181_42c7_a694_2191e06d524c.slice/crio-2d33747611f46671a177fac68f5c7baf9b7a316cb19aecec816db1af77e70758 WatchSource:0}: Error finding container 2d33747611f46671a177fac68f5c7baf9b7a316cb19aecec816db1af77e70758: Status 404 returned error can't find the container with id 2d33747611f46671a177fac68f5c7baf9b7a316cb19aecec816db1af77e70758 Sep 19 05:42:20.384090 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.384069 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b2b8g\"" Sep 19 05:42:20.392650 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.392629 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:42:20.460722 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.460317 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:20.513093 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.513069 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 05:42:20.518002 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.517963 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.521594 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.521426 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 19 05:42:20.522042 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.521829 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 19 05:42:20.522042 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.521861 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 19 05:42:20.522207 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.522105 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 19 05:42:20.522207 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.522160 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6zt88\"" Sep 19 05:42:20.522500 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.522346 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 19 05:42:20.522500 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.522376 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 19 05:42:20.522630 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.522596 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 19 05:42:20.522894 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.522879 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 19 05:42:20.522973 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.522894 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 19 05:42:20.538094 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.538029 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 05:42:20.564178 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.564116 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:20.568051 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.567900 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s9kmn" event={"ID":"8d49d458-d0eb-4619-9b41-2aa12b664541","Type":"ContainerStarted","Data":"a35b46ac4877749a11f8e8fdc0addb77ea21db0dbea4209e6c3752374f41c4a5"} Sep 19 05:42:20.570432 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.570371 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8947n" event={"ID":"ff709368-cadb-49c8-8bf1-eb0c3938922c","Type":"ContainerStarted","Data":"d11b145a89e93c16907128d34d0a9c1312fe071ca4788397c9361fefa0216fcd"} Sep 19 05:42:20.572869 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.572844 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" event={"ID":"3335bcd0-e181-42c7-a694-2191e06d524c","Type":"ContainerStarted","Data":"8275f634e001054fdc41be1af9e95dc1e9599d77714dae0dc9868bd878752eac"} Sep 19 05:42:20.572989 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.572899 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" event={"ID":"3335bcd0-e181-42c7-a694-2191e06d524c","Type":"ContainerStarted","Data":"6ddfcae6263815b3d24cdb0dcd2e0d9558450a525b4fc09b5059e5653abc7820"} Sep 19 05:42:20.572989 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.572928 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" event={"ID":"3335bcd0-e181-42c7-a694-2191e06d524c","Type":"ContainerStarted","Data":"2d33747611f46671a177fac68f5c7baf9b7a316cb19aecec816db1af77e70758"} Sep 19 05:42:20.574217 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.574193 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" event={"ID":"07b5c728-7106-4bab-87f1-007e05822927","Type":"ContainerStarted","Data":"8a39794ceebd3a404c22d7907caf82c59e7019e8abc7529980cc079f4c26b5cd"} Sep 19 05:42:20.644937 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.644849 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd88ec05-cf6b-4480-8469-d997b0453cb8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.644937 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.644908 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.644937 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.644937 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd88ec05-cf6b-4480-8469-d997b0453cb8-config-out\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.645211 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.644972 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd88ec05-cf6b-4480-8469-d997b0453cb8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.645211 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.645058 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-web-config\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.645211 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.645096 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.645211 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.645131 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqcc7\" (UniqueName: \"kubernetes.io/projected/bd88ec05-cf6b-4480-8469-d997b0453cb8-kube-api-access-hqcc7\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.645211 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.645172 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.645211 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.645195 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.645511 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.645222 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-config-volume\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.645511 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.645302 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd88ec05-cf6b-4480-8469-d997b0453cb8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.645511 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.645347 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd88ec05-cf6b-4480-8469-d997b0453cb8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.645511 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.645449 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.685990 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.685955 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pnp4s"] Sep 19 05:42:20.746410 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746351 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd88ec05-cf6b-4480-8469-d997b0453cb8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746570 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746437 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd88ec05-cf6b-4480-8469-d997b0453cb8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746570 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746463 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746570 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746494 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd88ec05-cf6b-4480-8469-d997b0453cb8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746570 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746516 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746570 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746548 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd88ec05-cf6b-4480-8469-d997b0453cb8-config-out\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746846 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746584 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd88ec05-cf6b-4480-8469-d997b0453cb8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746846 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746626 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-web-config\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746846 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746651 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746846 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746681 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqcc7\" (UniqueName: \"kubernetes.io/projected/bd88ec05-cf6b-4480-8469-d997b0453cb8-kube-api-access-hqcc7\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746846 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746705 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746846 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746725 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746846 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746748 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-config-volume\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.746846 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.746832 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bd88ec05-cf6b-4480-8469-d997b0453cb8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.747572 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.747497 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd88ec05-cf6b-4480-8469-d997b0453cb8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.747572 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.747554 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd88ec05-cf6b-4480-8469-d997b0453cb8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.749899 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.749868 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.750001 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.749919 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.750124 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.750101 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.750481 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.750437 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.750668 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.750527 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.750668 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.750650 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-config-volume\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.751111 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.751091 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd88ec05-cf6b-4480-8469-d997b0453cb8-config-out\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.751371 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.751355 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd88ec05-cf6b-4480-8469-d997b0453cb8-web-config\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.752312 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.752294 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd88ec05-cf6b-4480-8469-d997b0453cb8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.761030 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.760993 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqcc7\" (UniqueName: \"kubernetes.io/projected/bd88ec05-cf6b-4480-8469-d997b0453cb8-kube-api-access-hqcc7\") pod \"alertmanager-main-0\" (UID: \"bd88ec05-cf6b-4480-8469-d997b0453cb8\") " pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.769800 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.769752 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:20.841011 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.840966 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 19 05:42:20.908042 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:20.907973 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27fb3423_d426_4d0c_b117_439667e2be79.slice/crio-7273ddc66fa593c3ebf28bef61f0bd889aab6c5420725efe36a002fcde114047 WatchSource:0}: Error finding container 7273ddc66fa593c3ebf28bef61f0bd889aab6c5420725efe36a002fcde114047: Status 404 returned error can't find the container with id 7273ddc66fa593c3ebf28bef61f0bd889aab6c5420725efe36a002fcde114047 Sep 19 05:42:20.948298 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.948258 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-tls\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:20.950805 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:20.950771 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e87e7c2f-8946-40ef-8198-2daac6a33144-node-exporter-tls\") pod \"node-exporter-cjrpq\" (UID: \"e87e7c2f-8946-40ef-8198-2daac6a33144\") " pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:21.015773 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:21.015738 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cjrpq" Sep 19 05:42:21.029271 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:21.029246 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode87e7c2f_8946_40ef_8198_2daac6a33144.slice/crio-d296855d8980c47286d7f70a4393a2c54402259b7df81e064621172a9c1a3705 WatchSource:0}: Error finding container d296855d8980c47286d7f70a4393a2c54402259b7df81e064621172a9c1a3705: Status 404 returned error can't find the container with id d296855d8980c47286d7f70a4393a2c54402259b7df81e064621172a9c1a3705 Sep 19 05:42:21.192757 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:21.192726 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 19 05:42:21.202609 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:21.202577 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd88ec05_cf6b_4480_8469_d997b0453cb8.slice/crio-678526faabb2a6bf9c3fba145b005ad3595555cbd9c343db3ded12bc01b493a8 WatchSource:0}: Error finding container 678526faabb2a6bf9c3fba145b005ad3595555cbd9c343db3ded12bc01b493a8: Status 404 returned error can't find the container with id 678526faabb2a6bf9c3fba145b005ad3595555cbd9c343db3ded12bc01b493a8 Sep 19 05:42:21.581949 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:21.581858 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjrpq" event={"ID":"e87e7c2f-8946-40ef-8198-2daac6a33144","Type":"ContainerStarted","Data":"d296855d8980c47286d7f70a4393a2c54402259b7df81e064621172a9c1a3705"} Sep 19 05:42:21.586931 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:21.586898 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" event={"ID":"07b5c728-7106-4bab-87f1-007e05822927","Type":"ContainerStarted","Data":"dbeadaef452e0e7b0c3d1a6e862074a99439287e3813f88ab5265643060fa60c"} Sep 19 05:42:21.587077 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:21.586966 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" event={"ID":"07b5c728-7106-4bab-87f1-007e05822927","Type":"ContainerStarted","Data":"62590d44d9548a6abb706cd992b7bdaecce491c4d767a942bb1389d4b7326e38"} Sep 19 05:42:21.587077 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:21.586998 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" event={"ID":"07b5c728-7106-4bab-87f1-007e05822927","Type":"ContainerStarted","Data":"83bc9ecedf7b1dfc452cf40988ffc45587193fb0e12a575495fe0890fa2109a1"} Sep 19 05:42:21.589274 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:21.589222 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd88ec05-cf6b-4480-8469-d997b0453cb8","Type":"ContainerStarted","Data":"678526faabb2a6bf9c3fba145b005ad3595555cbd9c343db3ded12bc01b493a8"} Sep 19 05:42:21.590667 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:21.590616 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pnp4s" event={"ID":"27fb3423-d426-4d0c-b117-439667e2be79","Type":"ContainerStarted","Data":"7273ddc66fa593c3ebf28bef61f0bd889aab6c5420725efe36a002fcde114047"} Sep 19 05:42:22.047021 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.046908 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:22.150800 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.150713 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:22.191639 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.191214 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:22.191639 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.191330 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:22.199480 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.199455 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:22.232752 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.232693 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-s4gwv" podStartSLOduration=2.016196403 podStartE2EDuration="3.232675604s" totalImagesPullingTime="1.216479201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:21.611958233 +0000 UTC m=+69.605185766" watchObservedRunningTime="2026-09-19 05:42:22.232675604 +0000 UTC m=+70.225903129" Sep 19 05:42:22.264377 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.264312 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:22.363035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.362673 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:22.423752 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.423627 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-75f8b449ff-l4x2r"] Sep 19 05:42:22.452301 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.452222 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-75f8b449ff-l4x2r"] Sep 19 05:42:22.452520 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.452364 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.456740 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.456266 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-31dhusjufb6v5\"" Sep 19 05:42:22.456740 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.456393 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 19 05:42:22.456740 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.456519 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-td8rd\"" Sep 19 05:42:22.457142 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.457119 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 19 05:42:22.457269 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.457170 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 19 05:42:22.457269 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.457119 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 19 05:42:22.457554 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.457441 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 19 05:42:22.562671 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.562626 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.562809 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.562680 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.562809 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.562725 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.562809 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.562781 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6wl9\" (UniqueName: \"kubernetes.io/projected/67f88c7a-c35f-46cb-83f6-2ce328bc297f-kube-api-access-p6wl9\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.562809 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.562814 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67f88c7a-c35f-46cb-83f6-2ce328bc297f-metrics-client-ca\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.563083 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.562858 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-tls\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.563083 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.562888 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.563083 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.562917 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-grpc-tls\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.604194 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.603981 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:42:22.664781 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.663987 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.664781 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.664114 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6wl9\" (UniqueName: \"kubernetes.io/projected/67f88c7a-c35f-46cb-83f6-2ce328bc297f-kube-api-access-p6wl9\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.664781 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.664148 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67f88c7a-c35f-46cb-83f6-2ce328bc297f-metrics-client-ca\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.664781 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.664226 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-tls\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.664781 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.664253 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.664781 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.664273 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-grpc-tls\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.665195 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.665124 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67f88c7a-c35f-46cb-83f6-2ce328bc297f-metrics-client-ca\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.665252 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.665204 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.665252 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.665241 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.669729 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.669642 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.669886 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.669726 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.670879 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.670828 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.672779 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.672741 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-tls\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.673605 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.673560 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-grpc-tls\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.674771 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.674727 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/67f88c7a-c35f-46cb-83f6-2ce328bc297f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.676793 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.676707 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6wl9\" (UniqueName: \"kubernetes.io/projected/67f88c7a-c35f-46cb-83f6-2ce328bc297f-kube-api-access-p6wl9\") pod \"thanos-querier-75f8b449ff-l4x2r\" (UID: \"67f88c7a-c35f-46cb-83f6-2ce328bc297f\") " pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:22.695617 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.695582 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fc5999b4b-nrk72"] Sep 19 05:42:22.766337 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:22.766300 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:23.272905 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.272803 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:23.342610 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.342571 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:23.438536 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.438425 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6b46b4776c-8zmtc"] Sep 19 05:42:23.442433 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.442281 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.446217 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.446191 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 19 05:42:23.446217 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.446191 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-mvwnz\"" Sep 19 05:42:23.446424 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.446259 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 19 05:42:23.447318 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.447295 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d0l0squ0jv4i8\"" Sep 19 05:42:23.447439 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.447352 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 19 05:42:23.447439 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.447367 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 19 05:42:23.458484 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.458462 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6b46b4776c-8zmtc"] Sep 19 05:42:23.572979 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.572884 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-secret-metrics-server-client-certs\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.572979 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.572941 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-client-ca-bundle\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.573211 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.573002 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-audit-log\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.573211 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.573100 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-secret-metrics-server-tls\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.573211 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.573160 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87tbb\" (UniqueName: \"kubernetes.io/projected/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-kube-api-access-87tbb\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.573211 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.573193 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-metrics-server-audit-profiles\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.573429 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.573253 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.674621 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.674576 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-secret-metrics-server-client-certs\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.675097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.674672 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-client-ca-bundle\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.675097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.674710 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-audit-log\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.675097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.674792 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-secret-metrics-server-tls\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.675097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.674897 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87tbb\" (UniqueName: \"kubernetes.io/projected/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-kube-api-access-87tbb\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.675097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.674938 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-metrics-server-audit-profiles\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.675097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.674971 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.675412 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.675164 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-audit-log\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.676027 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.675997 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.676150 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.676073 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-metrics-server-audit-profiles\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.677915 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.677892 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-secret-metrics-server-tls\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.678026 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.677926 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-client-ca-bundle\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.678026 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.677976 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-secret-metrics-server-client-certs\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.686811 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.686789 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87tbb\" (UniqueName: \"kubernetes.io/projected/7c778cac-fa09-42dd-966c-8dd75ec2e1b8-kube-api-access-87tbb\") pod \"metrics-server-6b46b4776c-8zmtc\" (UID: \"7c778cac-fa09-42dd-966c-8dd75ec2e1b8\") " pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.755608 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.755570 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:23.931006 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.930965 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:23.950965 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.950938 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-7c4js"] Sep 19 05:42:23.954125 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.954101 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" Sep 19 05:42:23.957604 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.957562 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 19 05:42:23.957718 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.957640 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rnphk\"" Sep 19 05:42:23.970619 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:23.970594 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-7c4js"] Sep 19 05:42:24.028239 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.028187 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:24.078820 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.078785 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2fh2\" (UniqueName: \"kubernetes.io/projected/0d5ae841-0fb7-4036-81a7-8619412a5e3f-kube-api-access-v2fh2\") pod \"monitoring-plugin-5984459ff8-7c4js\" (UID: \"0d5ae841-0fb7-4036-81a7-8619412a5e3f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" Sep 19 05:42:24.078984 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.078837 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d5ae841-0fb7-4036-81a7-8619412a5e3f-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-7c4js\" (UID: \"0d5ae841-0fb7-4036-81a7-8619412a5e3f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" Sep 19 05:42:24.180160 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.180121 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2fh2\" (UniqueName: \"kubernetes.io/projected/0d5ae841-0fb7-4036-81a7-8619412a5e3f-kube-api-access-v2fh2\") pod \"monitoring-plugin-5984459ff8-7c4js\" (UID: \"0d5ae841-0fb7-4036-81a7-8619412a5e3f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" Sep 19 05:42:24.180336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.180195 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d5ae841-0fb7-4036-81a7-8619412a5e3f-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-7c4js\" (UID: \"0d5ae841-0fb7-4036-81a7-8619412a5e3f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" Sep 19 05:42:24.180336 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:24.180322 2584 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 19 05:42:24.180447 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:24.180422 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d5ae841-0fb7-4036-81a7-8619412a5e3f-monitoring-plugin-cert podName:0d5ae841-0fb7-4036-81a7-8619412a5e3f nodeName:}" failed. No retries permitted until 2026-09-19 05:42:24.68037672 +0000 UTC m=+72.673604235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/0d5ae841-0fb7-4036-81a7-8619412a5e3f-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-7c4js" (UID: "0d5ae841-0fb7-4036-81a7-8619412a5e3f") : secret "monitoring-plugin-cert" not found Sep 19 05:42:24.210936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.210870 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2fh2\" (UniqueName: \"kubernetes.io/projected/0d5ae841-0fb7-4036-81a7-8619412a5e3f-kube-api-access-v2fh2\") pod \"monitoring-plugin-5984459ff8-7c4js\" (UID: \"0d5ae841-0fb7-4036-81a7-8619412a5e3f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" Sep 19 05:42:24.328853 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.328804 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:24.543800 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.543724 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk"] Sep 19 05:42:24.547313 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.547273 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.552340 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.552314 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-blmnt\"" Sep 19 05:42:24.552511 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.552343 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 19 05:42:24.552805 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.552787 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 19 05:42:24.552919 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.552840 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 19 05:42:24.553603 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.553442 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 19 05:42:24.553603 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.553455 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 19 05:42:24.558221 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.558201 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 19 05:42:24.563708 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.563683 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:24.564361 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.564342 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk"] Sep 19 05:42:24.585105 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.585069 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/961b15e0-e8e1-429a-8f40-33ad81888eb4-federate-client-tls\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.585230 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.585118 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/961b15e0-e8e1-429a-8f40-33ad81888eb4-serving-certs-ca-bundle\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.585230 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.585182 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/961b15e0-e8e1-429a-8f40-33ad81888eb4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.585230 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.585219 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/961b15e0-e8e1-429a-8f40-33ad81888eb4-metrics-client-ca\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.585413 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.585265 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmjdl\" (UniqueName: \"kubernetes.io/projected/961b15e0-e8e1-429a-8f40-33ad81888eb4-kube-api-access-qmjdl\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.585413 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.585304 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/961b15e0-e8e1-429a-8f40-33ad81888eb4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.585413 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.585334 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/961b15e0-e8e1-429a-8f40-33ad81888eb4-telemeter-client-tls\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.585533 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.585477 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/961b15e0-e8e1-429a-8f40-33ad81888eb4-secret-telemeter-client\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.647553 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.647517 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:24.686082 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.686042 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/961b15e0-e8e1-429a-8f40-33ad81888eb4-telemeter-client-tls\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.686552 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.686146 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/961b15e0-e8e1-429a-8f40-33ad81888eb4-secret-telemeter-client\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.686552 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.686189 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/961b15e0-e8e1-429a-8f40-33ad81888eb4-federate-client-tls\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.686552 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.686221 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/961b15e0-e8e1-429a-8f40-33ad81888eb4-serving-certs-ca-bundle\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.686552 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.686268 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d5ae841-0fb7-4036-81a7-8619412a5e3f-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-7c4js\" (UID: \"0d5ae841-0fb7-4036-81a7-8619412a5e3f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" Sep 19 05:42:24.686552 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.686291 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/961b15e0-e8e1-429a-8f40-33ad81888eb4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.686552 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.686339 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/961b15e0-e8e1-429a-8f40-33ad81888eb4-metrics-client-ca\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.686552 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.686431 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmjdl\" (UniqueName: \"kubernetes.io/projected/961b15e0-e8e1-429a-8f40-33ad81888eb4-kube-api-access-qmjdl\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.686552 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.686502 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/961b15e0-e8e1-429a-8f40-33ad81888eb4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.687581 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.687157 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/961b15e0-e8e1-429a-8f40-33ad81888eb4-serving-certs-ca-bundle\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.687581 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.687555 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/961b15e0-e8e1-429a-8f40-33ad81888eb4-metrics-client-ca\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.688098 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.688065 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/961b15e0-e8e1-429a-8f40-33ad81888eb4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.689868 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.689457 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/961b15e0-e8e1-429a-8f40-33ad81888eb4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.689868 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.689624 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/961b15e0-e8e1-429a-8f40-33ad81888eb4-federate-client-tls\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.690008 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.689981 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/961b15e0-e8e1-429a-8f40-33ad81888eb4-telemeter-client-tls\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.690253 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.690232 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/961b15e0-e8e1-429a-8f40-33ad81888eb4-secret-telemeter-client\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.691097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.691076 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d5ae841-0fb7-4036-81a7-8619412a5e3f-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-7c4js\" (UID: \"0d5ae841-0fb7-4036-81a7-8619412a5e3f\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" Sep 19 05:42:24.705692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.705669 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmjdl\" (UniqueName: \"kubernetes.io/projected/961b15e0-e8e1-429a-8f40-33ad81888eb4-kube-api-access-qmjdl\") pod \"telemeter-client-86b9b79cb5-g7fvk\" (UID: \"961b15e0-e8e1-429a-8f40-33ad81888eb4\") " pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.858977 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.858944 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" Sep 19 05:42:24.865250 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:24.865224 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" Sep 19 05:42:25.060660 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.060624 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:25.114186 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.114103 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77dfff459b-cj4b5"] Sep 19 05:42:25.117450 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.117387 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.136124 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.136096 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77dfff459b-cj4b5"] Sep 19 05:42:25.191936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.191781 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-serving-cert\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.191936 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.191827 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbkvw\" (UniqueName: \"kubernetes.io/projected/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-kube-api-access-wbkvw\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.192135 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.191955 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-service-ca\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.192135 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.192039 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-config\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.192135 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.192061 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-trusted-ca-bundle\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.192135 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.192121 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-oauth-config\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.192276 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.192164 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-oauth-serving-cert\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.244865 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.244828 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:25.293263 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.293227 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-oauth-config\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.293495 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.293277 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-oauth-serving-cert\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.293495 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.293312 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-serving-cert\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.293495 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.293332 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbkvw\" (UniqueName: \"kubernetes.io/projected/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-kube-api-access-wbkvw\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.293495 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.293383 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-service-ca\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.293495 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.293481 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-config\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.293741 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.293712 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-trusted-ca-bundle\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.294186 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.294160 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-oauth-serving-cert\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.294186 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.294175 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-config\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.294392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.294362 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-service-ca\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.294699 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.294677 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-trusted-ca-bundle\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.296117 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.296095 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-serving-cert\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.296320 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.296300 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-oauth-config\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.324449 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.324389 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbkvw\" (UniqueName: \"kubernetes.io/projected/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-kube-api-access-wbkvw\") pod \"console-77dfff459b-cj4b5\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.429870 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.429773 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:25.457730 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.457690 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:25.684073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.683991 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 05:42:25.684073 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.684046 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:25.690098 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.690071 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.696332 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.696301 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 19 05:42:25.697238 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.697165 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 19 05:42:25.697361 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.697165 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2hsj54hmjc9os\"" Sep 19 05:42:25.697361 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.697354 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 19 05:42:25.698381 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.698140 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 19 05:42:25.698381 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.698243 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 19 05:42:25.698381 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.698332 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 19 05:42:25.699718 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.698778 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 19 05:42:25.701143 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.700049 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 19 05:42:25.701143 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.700256 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qnr45\"" Sep 19 05:42:25.701143 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.700293 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 19 05:42:25.701143 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.700712 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 19 05:42:25.702780 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.702381 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 19 05:42:25.702780 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.702614 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 19 05:42:25.709089 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.709068 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 19 05:42:25.709629 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.709607 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 19 05:42:25.721527 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.721507 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 05:42:25.770373 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.770332 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:25.801867 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.801826 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-config\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.801884 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.801917 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.801948 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.801964 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.801983 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802007 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dbf49520-2af6-4f63-97ce-35d666e85062-config-out\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802028 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802290 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802042 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802290 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802066 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802290 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802135 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802290 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802169 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802290 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802215 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdv4d\" (UniqueName: \"kubernetes.io/projected/dbf49520-2af6-4f63-97ce-35d666e85062-kube-api-access-gdv4d\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802290 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802247 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802290 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802279 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802559 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802303 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802559 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802327 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802559 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802346 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-web-config\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802559 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802393 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dbf49520-2af6-4f63-97ce-35d666e85062-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.802559 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.802462 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903191 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903151 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903371 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903203 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903371 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903237 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdv4d\" (UniqueName: \"kubernetes.io/projected/dbf49520-2af6-4f63-97ce-35d666e85062-kube-api-access-gdv4d\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903371 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903263 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903371 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903297 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903371 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903323 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903371 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903358 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903388 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-web-config\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903444 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dbf49520-2af6-4f63-97ce-35d666e85062-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903469 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903519 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-config\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903543 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903565 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903607 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903632 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903662 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.903692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903689 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dbf49520-2af6-4f63-97ce-35d666e85062-config-out\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.904127 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903715 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.904127 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903738 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.904127 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.903774 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.906103 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.905964 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.906368 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.906247 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.906710 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.906527 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.907350 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.907031 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.907350 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.907342 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.907997 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.907967 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.908288 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.908264 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.908824 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.908801 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dbf49520-2af6-4f63-97ce-35d666e85062-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.910698 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.910675 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.911636 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.911483 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-web-config\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.911636 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.911513 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.911636 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.911585 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.912097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.911957 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.912097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.911989 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-config\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.912097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.911991 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.912097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.912049 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dbf49520-2af6-4f63-97ce-35d666e85062-config-out\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.912479 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.912163 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.912479 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.912256 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dbf49520-2af6-4f63-97ce-35d666e85062-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.912856 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.912833 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dbf49520-2af6-4f63-97ce-35d666e85062-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:25.919527 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:25.919506 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdv4d\" (UniqueName: \"kubernetes.io/projected/dbf49520-2af6-4f63-97ce-35d666e85062-kube-api-access-gdv4d\") pod \"prometheus-k8s-0\" (UID: \"dbf49520-2af6-4f63-97ce-35d666e85062\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:26.016568 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.016044 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:26.230777 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.230683 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-75f8b449ff-l4x2r"] Sep 19 05:42:26.236512 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:26.236140 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67f88c7a_c35f_46cb_83f6_2ce328bc297f.slice/crio-64bc7bc73d83fc1ca2bf94a84da1238d47070c7fb92ada7da8aa60054517ebd6 WatchSource:0}: Error finding container 64bc7bc73d83fc1ca2bf94a84da1238d47070c7fb92ada7da8aa60054517ebd6: Status 404 returned error can't find the container with id 64bc7bc73d83fc1ca2bf94a84da1238d47070c7fb92ada7da8aa60054517ebd6 Sep 19 05:42:26.271881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.271856 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6b46b4776c-8zmtc"] Sep 19 05:42:26.288127 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.287025 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77dfff459b-cj4b5"] Sep 19 05:42:26.290756 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.289447 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk"] Sep 19 05:42:26.291678 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.291656 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-7c4js"] Sep 19 05:42:26.322630 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:26.322587 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e7ccd71_fe2a_4b18_baec_b17dff3899c2.slice/crio-b87cc4fa662bb9736d1a85fc8d016a0d95d542e70727e5ec02d3dff0139cdf7a WatchSource:0}: Error finding container b87cc4fa662bb9736d1a85fc8d016a0d95d542e70727e5ec02d3dff0139cdf7a: Status 404 returned error can't find the container with id b87cc4fa662bb9736d1a85fc8d016a0d95d542e70727e5ec02d3dff0139cdf7a Sep 19 05:42:26.323301 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:26.323275 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d5ae841_0fb7_4036_81a7_8619412a5e3f.slice/crio-55ddc24762fb6ef8d4261142aca1a2321f76c42b8cb14b3b433cfc18a520f2fd WatchSource:0}: Error finding container 55ddc24762fb6ef8d4261142aca1a2321f76c42b8cb14b3b433cfc18a520f2fd: Status 404 returned error can't find the container with id 55ddc24762fb6ef8d4261142aca1a2321f76c42b8cb14b3b433cfc18a520f2fd Sep 19 05:42:26.323790 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:26.323764 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod961b15e0_e8e1_429a_8f40_33ad81888eb4.slice/crio-68f22b4bde8debf7976a90467ec5aa197be89fb1adeb6c431389a4a7b4a4ac8f WatchSource:0}: Error finding container 68f22b4bde8debf7976a90467ec5aa197be89fb1adeb6c431389a4a7b4a4ac8f: Status 404 returned error can't find the container with id 68f22b4bde8debf7976a90467ec5aa197be89fb1adeb6c431389a4a7b4a4ac8f Sep 19 05:42:26.374981 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.374953 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 05:42:26.377942 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:26.377913 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf49520_2af6_4f63_97ce_35d666e85062.slice/crio-78cc69e0002156f5230c6b2120585e9352edb53b5be309ccd8456724d9b1a719 WatchSource:0}: Error finding container 78cc69e0002156f5230c6b2120585e9352edb53b5be309ccd8456724d9b1a719: Status 404 returned error can't find the container with id 78cc69e0002156f5230c6b2120585e9352edb53b5be309ccd8456724d9b1a719 Sep 19 05:42:26.618142 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.618051 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" event={"ID":"0d5ae841-0fb7-4036-81a7-8619412a5e3f","Type":"ContainerStarted","Data":"55ddc24762fb6ef8d4261142aca1a2321f76c42b8cb14b3b433cfc18a520f2fd"} Sep 19 05:42:26.619415 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.619377 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" event={"ID":"67f88c7a-c35f-46cb-83f6-2ce328bc297f","Type":"ContainerStarted","Data":"64bc7bc73d83fc1ca2bf94a84da1238d47070c7fb92ada7da8aa60054517ebd6"} Sep 19 05:42:26.620889 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.620864 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s9kmn" event={"ID":"8d49d458-d0eb-4619-9b41-2aa12b664541","Type":"ContainerStarted","Data":"2a4621f16936734ae3ea65f4c84ef1ee291ee2577ab9130f375238265d60d86f"} Sep 19 05:42:26.622442 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.622394 2584 generic.go:356] "Generic (PLEG): container finished" podID="e87e7c2f-8946-40ef-8198-2daac6a33144" containerID="53438044aa4725f854c3c70a0c83269b25d0ddd535bc012a73e338182bb058a7" exitCode=0 Sep 19 05:42:26.622523 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.622492 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjrpq" event={"ID":"e87e7c2f-8946-40ef-8198-2daac6a33144","Type":"ContainerDied","Data":"53438044aa4725f854c3c70a0c83269b25d0ddd535bc012a73e338182bb058a7"} Sep 19 05:42:26.624492 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.624455 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8947n" event={"ID":"ff709368-cadb-49c8-8bf1-eb0c3938922c","Type":"ContainerStarted","Data":"5954890d3d40ac5c1af36786f575b37d1c85b9dac817a71204d41d9f8d4f7f86"} Sep 19 05:42:26.624585 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.624503 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8947n" event={"ID":"ff709368-cadb-49c8-8bf1-eb0c3938922c","Type":"ContainerStarted","Data":"9d77a41144df876210e4a174a7c4c072a9bcf58db0a75b331c8e7c344cf6fce0"} Sep 19 05:42:26.626734 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.626710 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" event={"ID":"3335bcd0-e181-42c7-a694-2191e06d524c","Type":"ContainerStarted","Data":"7be43aa62f7ac0b1bf26cb3340fef1ef20932a3a27094bece3608d4ee94c39ed"} Sep 19 05:42:26.628390 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.628366 2584 generic.go:356] "Generic (PLEG): container finished" podID="bd88ec05-cf6b-4480-8469-d997b0453cb8" containerID="d7ad2ab9943e5205db2bbcd25ef8db3bfbac955133f362277949e5154fad6032" exitCode=0 Sep 19 05:42:26.628489 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.628422 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd88ec05-cf6b-4480-8469-d997b0453cb8","Type":"ContainerDied","Data":"d7ad2ab9943e5205db2bbcd25ef8db3bfbac955133f362277949e5154fad6032"} Sep 19 05:42:26.632662 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.632643 2584 generic.go:356] "Generic (PLEG): container finished" podID="dbf49520-2af6-4f63-97ce-35d666e85062" containerID="4753666b15df29af2440c48d1fac8dc259703e4212a78b52479e9d56f552e30d" exitCode=0 Sep 19 05:42:26.632771 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.632670 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbf49520-2af6-4f63-97ce-35d666e85062","Type":"ContainerDied","Data":"4753666b15df29af2440c48d1fac8dc259703e4212a78b52479e9d56f552e30d"} Sep 19 05:42:26.632771 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.632751 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbf49520-2af6-4f63-97ce-35d666e85062","Type":"ContainerStarted","Data":"78cc69e0002156f5230c6b2120585e9352edb53b5be309ccd8456724d9b1a719"} Sep 19 05:42:26.634154 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.634133 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" event={"ID":"961b15e0-e8e1-429a-8f40-33ad81888eb4","Type":"ContainerStarted","Data":"68f22b4bde8debf7976a90467ec5aa197be89fb1adeb6c431389a4a7b4a4ac8f"} Sep 19 05:42:26.635856 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.635835 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dfff459b-cj4b5" event={"ID":"5e7ccd71-fe2a-4b18-baec-b17dff3899c2","Type":"ContainerStarted","Data":"c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103"} Sep 19 05:42:26.635953 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.635867 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dfff459b-cj4b5" event={"ID":"5e7ccd71-fe2a-4b18-baec-b17dff3899c2","Type":"ContainerStarted","Data":"b87cc4fa662bb9736d1a85fc8d016a0d95d542e70727e5ec02d3dff0139cdf7a"} Sep 19 05:42:26.637102 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.637073 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" event={"ID":"7c778cac-fa09-42dd-966c-8dd75ec2e1b8","Type":"ContainerStarted","Data":"42b60a4c85e7a02946b156c275095ddbadcc78bdca220dd70490897769df5df6"} Sep 19 05:42:26.638611 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.638587 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pnp4s" event={"ID":"27fb3423-d426-4d0c-b117-439667e2be79","Type":"ContainerStarted","Data":"d30301e8de517765151ab8c6b9aa654483e67bcc198edf734e20f5224858a3de"} Sep 19 05:42:26.638726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.638709 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:42:26.639524 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.639501 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:26.644312 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.644273 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-s9kmn" podStartSLOduration=67.044368219 podStartE2EDuration="1m12.644259606s" totalImagesPullingTime="5.599891387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 05:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:26.64330363 +0000 UTC m=+74.636531163" watchObservedRunningTime="2026-09-19 05:42:26.644259606 +0000 UTC m=+74.637487131" Sep 19 05:42:26.718488 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.718377 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-r8tjl" podStartSLOduration=3.922381326 podStartE2EDuration="7.718354884s" totalImagesPullingTime="3.795973558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:26.7168464 +0000 UTC m=+74.710073933" watchObservedRunningTime="2026-09-19 05:42:26.718354884 +0000 UTC m=+74.711582416" Sep 19 05:42:26.725287 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.725255 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:26.774642 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.773722 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8947n" podStartSLOduration=68.803797142 podStartE2EDuration="1m12.773698571s" totalImagesPullingTime="3.969901429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:26.771218989 +0000 UTC m=+74.764446521" watchObservedRunningTime="2026-09-19 05:42:26.773698571 +0000 UTC m=+74.766926103" Sep 19 05:42:26.789518 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.788666 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pnp4s" podStartSLOduration=67.73462815 podStartE2EDuration="1m12.788646336s" totalImagesPullingTime="5.054018186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:26.787927512 +0000 UTC m=+74.781155043" watchObservedRunningTime="2026-09-19 05:42:26.788646336 +0000 UTC m=+74.781873869" Sep 19 05:42:26.810308 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.810248 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77dfff459b-cj4b5" podStartSLOduration=1.8102286250000001 podStartE2EDuration="1.810228625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:26.809505448 +0000 UTC m=+74.802732979" watchObservedRunningTime="2026-09-19 05:42:26.810228625 +0000 UTC m=+74.803456156" Sep 19 05:42:26.892833 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.892789 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:26.970065 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:26.969926 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:27.663000 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:27.662839 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjrpq" event={"ID":"e87e7c2f-8946-40ef-8198-2daac6a33144","Type":"ContainerStarted","Data":"c1d1c4f1d00f1944934446c7003cf48d26f9cc1385035675b25c36ca3ff9d9c3"} Sep 19 05:42:27.663000 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:27.662929 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cjrpq" event={"ID":"e87e7c2f-8946-40ef-8198-2daac6a33144","Type":"ContainerStarted","Data":"c765f72ba1bbf3138f1be3b1bdf81d454ff67f0f5f45f2d03e23e8d692f64497"} Sep 19 05:42:27.687313 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:27.687225 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cjrpq" podStartSLOduration=3.684954666 podStartE2EDuration="8.687202619s" totalImagesPullingTime="4.909613798s" totalInitContainerRuntime="92.634155ms" isStatefulPod=true podCreationTimestamp="2026-09-19 05:42:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:27.686188257 +0000 UTC m=+75.679415791" watchObservedRunningTime="2026-09-19 05:42:27.687202619 +0000 UTC m=+75.680430152" Sep 19 05:42:28.488601 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:28.488562 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:28.557048 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:28.557018 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:31.681918 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.681879 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" event={"ID":"7c778cac-fa09-42dd-966c-8dd75ec2e1b8","Type":"ContainerStarted","Data":"0f50f9854d1ad65369d65fd49985e851fb4bb411dfab2b62c06fc651b969b5a0"} Sep 19 05:42:31.687602 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.687565 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" event={"ID":"0d5ae841-0fb7-4036-81a7-8619412a5e3f","Type":"ContainerStarted","Data":"a8aa1504652b09e576358f13942e9856d15adb057213826a5cb5e325506425f9"} Sep 19 05:42:31.687772 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.687625 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" Sep 19 05:42:31.690351 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.690324 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" event={"ID":"67f88c7a-c35f-46cb-83f6-2ce328bc297f","Type":"ContainerStarted","Data":"a009d09d8c5200fd971fe526b3a2a0dea95cd2d4ea81b835262848be62381ca0"} Sep 19 05:42:31.690502 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.690417 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" event={"ID":"67f88c7a-c35f-46cb-83f6-2ce328bc297f","Type":"ContainerStarted","Data":"f34620613826deccb8daa71d406213dba7aa048abfcb747f218983d24952effe"} Sep 19 05:42:31.690502 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.690463 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" event={"ID":"67f88c7a-c35f-46cb-83f6-2ce328bc297f","Type":"ContainerStarted","Data":"ed2cbe8c2928baf14bd367c8424566d1b728dbd7341ac7a42a8776562cdb0bb9"} Sep 19 05:42:31.693210 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.693192 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" Sep 19 05:42:31.693391 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.693376 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd88ec05-cf6b-4480-8469-d997b0453cb8","Type":"ContainerStarted","Data":"513eec19fd4562b79c7ea870a8f0e60051c6cb72d1f930781156b59ae3390d03"} Sep 19 05:42:31.693456 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.693436 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd88ec05-cf6b-4480-8469-d997b0453cb8","Type":"ContainerStarted","Data":"f8b70810b0119021dd86c28d42b4e953de66334d13c3b4d2cbd9c003e13e2d8e"} Sep 19 05:42:31.693505 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.693476 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd88ec05-cf6b-4480-8469-d997b0453cb8","Type":"ContainerStarted","Data":"f21112eeb5f310cb38e18b4173e137d2e2520a6216b8e5f2555eb0d11b817437"} Sep 19 05:42:31.693552 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.693512 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd88ec05-cf6b-4480-8469-d997b0453cb8","Type":"ContainerStarted","Data":"71d577640aeb92c67d9c254ac2a293b13ddf83504561e0c6a93d289b8976afd4"} Sep 19 05:42:31.693552 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.693538 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd88ec05-cf6b-4480-8469-d997b0453cb8","Type":"ContainerStarted","Data":"853fdfb6fdf246ecd885f6c5d61aacccb6b63b32e78d48e50174474e36d2b97d"} Sep 19 05:42:31.696685 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.696664 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbf49520-2af6-4f63-97ce-35d666e85062","Type":"ContainerStarted","Data":"a9a2ec616ea4268a5b391a9098b9fe6b82d168880bede5339ad12b2ff09977dd"} Sep 19 05:42:31.696769 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.696712 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbf49520-2af6-4f63-97ce-35d666e85062","Type":"ContainerStarted","Data":"da1e1c46703ac37fa701be1a89d9d41ea167885477398b038c1a8488043a8fbf"} Sep 19 05:42:31.696769 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.696737 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbf49520-2af6-4f63-97ce-35d666e85062","Type":"ContainerStarted","Data":"e25df65c951f6e2e190b499cb523ef0ec02eb9d423d5771567e59a7691c5e655"} Sep 19 05:42:31.696769 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.696766 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbf49520-2af6-4f63-97ce-35d666e85062","Type":"ContainerStarted","Data":"68efe43580a45b683f418d5d645b13b256018cee174b5cdf4e796f22a082f66d"} Sep 19 05:42:31.696861 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.696788 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbf49520-2af6-4f63-97ce-35d666e85062","Type":"ContainerStarted","Data":"67144e90161b00111649def4dd21be1c1699a44407197410bc50c535eb234651"} Sep 19 05:42:31.696861 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.696811 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dbf49520-2af6-4f63-97ce-35d666e85062","Type":"ContainerStarted","Data":"8aedadc8753e20e043ac025a74c905a04ca1dec53929de528593a2c06ee0f728"} Sep 19 05:42:31.699325 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.699303 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" event={"ID":"961b15e0-e8e1-429a-8f40-33ad81888eb4","Type":"ContainerStarted","Data":"9f1cfbf0e04bc86ef27022fc87a7417b6737f9fcfaacb0b9caaf6a5a6695302b"} Sep 19 05:42:31.699513 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.699341 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" event={"ID":"961b15e0-e8e1-429a-8f40-33ad81888eb4","Type":"ContainerStarted","Data":"b772d9551daf93f55461061e18a40f0215fed7ba442488bbdf89854760d5d61c"} Sep 19 05:42:31.699513 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.699360 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" event={"ID":"961b15e0-e8e1-429a-8f40-33ad81888eb4","Type":"ContainerStarted","Data":"b84bde101bdff2fd2c34234b0bfb6b2294568c34a3858afd70384d2b68294cdf"} Sep 19 05:42:31.702222 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.702183 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" podStartSLOduration=3.972565415 podStartE2EDuration="8.702165711s" totalImagesPullingTime="4.729600296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:31.701108262 +0000 UTC m=+79.694335794" watchObservedRunningTime="2026-09-19 05:42:31.702165711 +0000 UTC m=+79.695393241" Sep 19 05:42:31.733077 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.728075 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-86b9b79cb5-g7fvk" podStartSLOduration=3.043561205 podStartE2EDuration="7.72805586s" totalImagesPullingTime="4.684494655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:31.725971512 +0000 UTC m=+79.719199054" watchObservedRunningTime="2026-09-19 05:42:31.72805586 +0000 UTC m=+79.721283392" Sep 19 05:42:31.744602 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.744558 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-7c4js" podStartSLOduration=4.063801992 podStartE2EDuration="8.744545152s" totalImagesPullingTime="4.68074316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:31.744339435 +0000 UTC m=+79.737567094" watchObservedRunningTime="2026-09-19 05:42:31.744545152 +0000 UTC m=+79.737772682" Sep 19 05:42:31.778154 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.778049 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.326095171 podStartE2EDuration="6.778028767s" totalImagesPullingTime="4.42944819s" totalInitContainerRuntime="22.485406ms" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:31.774460068 +0000 UTC m=+79.767687603" watchObservedRunningTime="2026-09-19 05:42:31.778028767 +0000 UTC m=+79.771256312" Sep 19 05:42:31.832265 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.832226 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77dfff459b-cj4b5"] Sep 19 05:42:31.871853 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.871812 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55c7f7cf49-k4vjb"] Sep 19 05:42:31.875348 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.875323 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:31.887739 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.887676 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c7f7cf49-k4vjb"] Sep 19 05:42:31.980187 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.979999 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-service-ca\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:31.980187 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.980061 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fg7j\" (UniqueName: \"kubernetes.io/projected/57c16f60-2e0f-4ab3-a847-01af496daaad-kube-api-access-9fg7j\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:31.980187 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.980103 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-oauth-serving-cert\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:31.980503 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.980221 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57c16f60-2e0f-4ab3-a847-01af496daaad-console-oauth-config\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:31.980503 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.980256 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-trusted-ca-bundle\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:31.980503 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.980318 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-console-config\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:31.980503 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:31.980360 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57c16f60-2e0f-4ab3-a847-01af496daaad-console-serving-cert\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.081902 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.081862 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57c16f60-2e0f-4ab3-a847-01af496daaad-console-oauth-config\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.082090 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.081914 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-trusted-ca-bundle\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.082090 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.081974 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-console-config\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.082090 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.082012 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57c16f60-2e0f-4ab3-a847-01af496daaad-console-serving-cert\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.082090 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.082049 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-service-ca\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.082090 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.082083 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fg7j\" (UniqueName: \"kubernetes.io/projected/57c16f60-2e0f-4ab3-a847-01af496daaad-kube-api-access-9fg7j\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.082328 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.082112 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-oauth-serving-cert\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.082966 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.082895 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-trusted-ca-bundle\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.082966 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.082898 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-oauth-serving-cert\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.083169 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.082968 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-console-config\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.083457 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.083388 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-service-ca\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.084892 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.084868 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57c16f60-2e0f-4ab3-a847-01af496daaad-console-oauth-config\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.085290 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.085269 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57c16f60-2e0f-4ab3-a847-01af496daaad-console-serving-cert\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.091841 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.091816 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fg7j\" (UniqueName: \"kubernetes.io/projected/57c16f60-2e0f-4ab3-a847-01af496daaad-kube-api-access-9fg7j\") pod \"console-55c7f7cf49-k4vjb\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.188666 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.188641 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:32.473953 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.473917 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c7f7cf49-k4vjb"] Sep 19 05:42:32.478085 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:42:32.478058 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57c16f60_2e0f_4ab3_a847_01af496daaad.slice/crio-19070d4e1d60014a537aa6a55c4462b10f3fd3c0af7f7d099ebc42461d1a14c3 WatchSource:0}: Error finding container 19070d4e1d60014a537aa6a55c4462b10f3fd3c0af7f7d099ebc42461d1a14c3: Status 404 returned error can't find the container with id 19070d4e1d60014a537aa6a55c4462b10f3fd3c0af7f7d099ebc42461d1a14c3 Sep 19 05:42:32.707843 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.707742 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bd88ec05-cf6b-4480-8469-d997b0453cb8","Type":"ContainerStarted","Data":"ed6500d116151c20a8aa491cded3cc4240b368d8f8c60a2fb9c1ab97c88f1b06"} Sep 19 05:42:32.711127 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.711095 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" event={"ID":"67f88c7a-c35f-46cb-83f6-2ce328bc297f","Type":"ContainerStarted","Data":"804e82e9784a75a61738d5b982ac77601397c408470ba5a4ecb5aa92a2e28f6f"} Sep 19 05:42:32.711256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.711167 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" event={"ID":"67f88c7a-c35f-46cb-83f6-2ce328bc297f","Type":"ContainerStarted","Data":"c1dba7fdfe60aac147a3bd35688f001014da2644c0795ab1822750b5d823f0c9"} Sep 19 05:42:32.711256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.711209 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" event={"ID":"67f88c7a-c35f-46cb-83f6-2ce328bc297f","Type":"ContainerStarted","Data":"e656a347cfc4d8163757dabb11706fb74d83a90e8c2a2d7f0e6ded6672af7607"} Sep 19 05:42:32.711256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.711252 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:32.713005 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.712981 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c7f7cf49-k4vjb" event={"ID":"57c16f60-2e0f-4ab3-a847-01af496daaad","Type":"ContainerStarted","Data":"82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9"} Sep 19 05:42:32.713115 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.713041 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c7f7cf49-k4vjb" event={"ID":"57c16f60-2e0f-4ab3-a847-01af496daaad","Type":"ContainerStarted","Data":"19070d4e1d60014a537aa6a55c4462b10f3fd3c0af7f7d099ebc42461d1a14c3"} Sep 19 05:42:32.766264 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.766210 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.014449 podStartE2EDuration="12.766193205s" totalImagesPullingTime="9.725581208s" totalInitContainerRuntime="26.162997ms" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:32.765952805 +0000 UTC m=+80.759180337" watchObservedRunningTime="2026-09-19 05:42:32.766193205 +0000 UTC m=+80.759420736" Sep 19 05:42:32.814873 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.814810 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" podStartSLOduration=5.261047419 podStartE2EDuration="10.814788351s" totalImagesPullingTime="5.553740932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:32.809878342 +0000 UTC m=+80.803105873" watchObservedRunningTime="2026-09-19 05:42:32.814788351 +0000 UTC m=+80.808015882" Sep 19 05:42:32.842274 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:32.842212 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55c7f7cf49-k4vjb" podStartSLOduration=1.842189673 podStartE2EDuration="1.842189673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:42:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:42:32.84133143 +0000 UTC m=+80.834558961" watchObservedRunningTime="2026-09-19 05:42:32.842189673 +0000 UTC m=+80.835417204" Sep 19 05:42:35.232215 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:35.232164 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:35.333797 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:35.333759 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:35.430485 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:35.430432 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:35.432084 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:35.432064 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:36.021284 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:36.021254 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:42:38.723916 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:38.723883 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-75f8b449ff-l4x2r" Sep 19 05:42:41.645908 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:41.645875 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:41.719119 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:41.719089 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:42.189479 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:42.189437 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:42.189685 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:42.189503 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:42.195349 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:42.195323 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:42.767115 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:42.767090 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:42:42.824698 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:42.824663 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66d6f87bd9-zxf26"] Sep 19 05:42:42.942986 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:42.942954 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:43.049652 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:43.049569 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:43.186165 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:43.186132 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:43.756276 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:43.756230 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:43.756465 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:43.756309 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:42:47.726007 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:47.725937 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fc5999b4b-nrk72" podUID="9300805f-f719-422b-a428-ae038d9e6a2c" containerName="console" containerID="cri-o://e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192" gracePeriod=15 Sep 19 05:42:47.983483 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:47.983424 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:48.042108 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.042080 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9300805f-f719-422b-a428-ae038d9e6a2c-console-oauth-config\") pod \"9300805f-f719-422b-a428-ae038d9e6a2c\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " Sep 19 05:42:48.042368 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.042125 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-oauth-serving-cert\") pod \"9300805f-f719-422b-a428-ae038d9e6a2c\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " Sep 19 05:42:48.042368 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.042193 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9300805f-f719-422b-a428-ae038d9e6a2c-console-serving-cert\") pod \"9300805f-f719-422b-a428-ae038d9e6a2c\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " Sep 19 05:42:48.042368 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.042272 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9774\" (UniqueName: \"kubernetes.io/projected/9300805f-f719-422b-a428-ae038d9e6a2c-kube-api-access-p9774\") pod \"9300805f-f719-422b-a428-ae038d9e6a2c\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " Sep 19 05:42:48.042368 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.042313 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-service-ca\") pod \"9300805f-f719-422b-a428-ae038d9e6a2c\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " Sep 19 05:42:48.042368 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.042337 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-console-config\") pod \"9300805f-f719-422b-a428-ae038d9e6a2c\" (UID: \"9300805f-f719-422b-a428-ae038d9e6a2c\") " Sep 19 05:42:48.042710 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.042618 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9300805f-f719-422b-a428-ae038d9e6a2c" (UID: "9300805f-f719-422b-a428-ae038d9e6a2c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:42:48.042710 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.042660 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-service-ca" (OuterVolumeSpecName: "service-ca") pod "9300805f-f719-422b-a428-ae038d9e6a2c" (UID: "9300805f-f719-422b-a428-ae038d9e6a2c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:42:48.042991 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.042964 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-console-config" (OuterVolumeSpecName: "console-config") pod "9300805f-f719-422b-a428-ae038d9e6a2c" (UID: "9300805f-f719-422b-a428-ae038d9e6a2c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:42:48.044694 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.044665 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9300805f-f719-422b-a428-ae038d9e6a2c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9300805f-f719-422b-a428-ae038d9e6a2c" (UID: "9300805f-f719-422b-a428-ae038d9e6a2c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:42:48.044815 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.044741 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9300805f-f719-422b-a428-ae038d9e6a2c-kube-api-access-p9774" (OuterVolumeSpecName: "kube-api-access-p9774") pod "9300805f-f719-422b-a428-ae038d9e6a2c" (UID: "9300805f-f719-422b-a428-ae038d9e6a2c"). InnerVolumeSpecName "kube-api-access-p9774". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:42:48.044815 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.044748 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9300805f-f719-422b-a428-ae038d9e6a2c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9300805f-f719-422b-a428-ae038d9e6a2c" (UID: "9300805f-f719-422b-a428-ae038d9e6a2c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:42:48.143590 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.143537 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9774\" (UniqueName: \"kubernetes.io/projected/9300805f-f719-422b-a428-ae038d9e6a2c-kube-api-access-p9774\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:48.143590 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.143585 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:48.143590 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.143595 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-console-config\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:48.143590 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.143602 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9300805f-f719-422b-a428-ae038d9e6a2c-console-oauth-config\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:48.143590 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.143613 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9300805f-f719-422b-a428-ae038d9e6a2c-oauth-serving-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:48.143892 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.143621 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9300805f-f719-422b-a428-ae038d9e6a2c-console-serving-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:48.791457 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.791420 2584 generic.go:356] "Generic (PLEG): container finished" podID="9300805f-f719-422b-a428-ae038d9e6a2c" containerID="e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192" exitCode=0 Sep 19 05:42:48.791937 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.791474 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc5999b4b-nrk72" Sep 19 05:42:48.791937 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.791510 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc5999b4b-nrk72" event={"ID":"9300805f-f719-422b-a428-ae038d9e6a2c","Type":"ContainerDied","Data":"e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192"} Sep 19 05:42:48.791937 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.791563 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc5999b4b-nrk72" event={"ID":"9300805f-f719-422b-a428-ae038d9e6a2c","Type":"ContainerDied","Data":"d9942b1f218a5c20f780049f9008a8d2036952656f497e3a221d06acf9732166"} Sep 19 05:42:48.791937 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.791584 2584 scope.go:117] "RemoveContainer" containerID="e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192" Sep 19 05:42:48.800194 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.800178 2584 scope.go:117] "RemoveContainer" containerID="e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192" Sep 19 05:42:48.800502 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:48.800480 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192\": container with ID starting with e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192 not found: ID does not exist" containerID="e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192" Sep 19 05:42:48.800557 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.800521 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192"} err="failed to get container status \"e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192\": rpc error: code = NotFound desc = could not find container \"e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192\": container with ID starting with e74effcd28f4f810b428cf00aa6eea899aff99f4c030bc3f95692cb97f7be192 not found: ID does not exist" Sep 19 05:42:48.808983 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.808961 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fc5999b4b-nrk72"] Sep 19 05:42:48.813626 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:48.813605 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fc5999b4b-nrk72"] Sep 19 05:42:50.266932 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:50.266899 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9300805f-f719-422b-a428-ae038d9e6a2c" path="/var/lib/kubelet/pods/9300805f-f719-422b-a428-ae038d9e6a2c/volumes" Sep 19 05:42:55.441286 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:55.441230 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:55.606321 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:55.606279 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:42:56.851916 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:56.851878 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77dfff459b-cj4b5" podUID="5e7ccd71-fe2a-4b18-baec-b17dff3899c2" containerName="console" containerID="cri-o://c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103" gracePeriod=15 Sep 19 05:42:57.128143 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.128079 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:57.223899 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.223860 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-config\") pod \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " Sep 19 05:42:57.223899 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.223905 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbkvw\" (UniqueName: \"kubernetes.io/projected/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-kube-api-access-wbkvw\") pod \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " Sep 19 05:42:57.224139 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.223942 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-trusted-ca-bundle\") pod \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " Sep 19 05:42:57.224139 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.223973 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-oauth-serving-cert\") pod \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " Sep 19 05:42:57.224139 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.224000 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-serving-cert\") pod \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " Sep 19 05:42:57.224139 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.224081 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-oauth-config\") pod \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " Sep 19 05:42:57.224139 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.224132 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-service-ca\") pod \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\" (UID: \"5e7ccd71-fe2a-4b18-baec-b17dff3899c2\") " Sep 19 05:42:57.224431 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.224304 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-config" (OuterVolumeSpecName: "console-config") pod "5e7ccd71-fe2a-4b18-baec-b17dff3899c2" (UID: "5e7ccd71-fe2a-4b18-baec-b17dff3899c2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:42:57.224493 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.224458 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-config\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:57.224493 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.224471 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5e7ccd71-fe2a-4b18-baec-b17dff3899c2" (UID: "5e7ccd71-fe2a-4b18-baec-b17dff3899c2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:42:57.224613 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.224460 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5e7ccd71-fe2a-4b18-baec-b17dff3899c2" (UID: "5e7ccd71-fe2a-4b18-baec-b17dff3899c2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:42:57.224712 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.224689 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-service-ca" (OuterVolumeSpecName: "service-ca") pod "5e7ccd71-fe2a-4b18-baec-b17dff3899c2" (UID: "5e7ccd71-fe2a-4b18-baec-b17dff3899c2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:42:57.226376 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.226339 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5e7ccd71-fe2a-4b18-baec-b17dff3899c2" (UID: "5e7ccd71-fe2a-4b18-baec-b17dff3899c2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:42:57.226376 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.226353 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-kube-api-access-wbkvw" (OuterVolumeSpecName: "kube-api-access-wbkvw") pod "5e7ccd71-fe2a-4b18-baec-b17dff3899c2" (UID: "5e7ccd71-fe2a-4b18-baec-b17dff3899c2"). InnerVolumeSpecName "kube-api-access-wbkvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:42:57.226595 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.226413 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5e7ccd71-fe2a-4b18-baec-b17dff3899c2" (UID: "5e7ccd71-fe2a-4b18-baec-b17dff3899c2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:42:57.325905 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.325873 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbkvw\" (UniqueName: \"kubernetes.io/projected/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-kube-api-access-wbkvw\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:57.325905 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.325903 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-trusted-ca-bundle\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:57.325905 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.325917 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-oauth-serving-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:57.326141 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.325929 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-serving-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:57.326141 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.325941 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-console-oauth-config\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:57.326141 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.325953 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e7ccd71-fe2a-4b18-baec-b17dff3899c2-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:42:57.667016 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.666987 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pnp4s" Sep 19 05:42:57.832152 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.832118 2584 generic.go:356] "Generic (PLEG): container finished" podID="5e7ccd71-fe2a-4b18-baec-b17dff3899c2" containerID="c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103" exitCode=0 Sep 19 05:42:57.832352 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.832163 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dfff459b-cj4b5" Sep 19 05:42:57.832352 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.832198 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dfff459b-cj4b5" event={"ID":"5e7ccd71-fe2a-4b18-baec-b17dff3899c2","Type":"ContainerDied","Data":"c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103"} Sep 19 05:42:57.832352 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.832251 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dfff459b-cj4b5" event={"ID":"5e7ccd71-fe2a-4b18-baec-b17dff3899c2","Type":"ContainerDied","Data":"b87cc4fa662bb9736d1a85fc8d016a0d95d542e70727e5ec02d3dff0139cdf7a"} Sep 19 05:42:57.832352 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.832277 2584 scope.go:117] "RemoveContainer" containerID="c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103" Sep 19 05:42:57.848128 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.848111 2584 scope.go:117] "RemoveContainer" containerID="c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103" Sep 19 05:42:57.848460 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:42:57.848435 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103\": container with ID starting with c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103 not found: ID does not exist" containerID="c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103" Sep 19 05:42:57.848513 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.848471 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103"} err="failed to get container status \"c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103\": rpc error: code = NotFound desc = could not find container \"c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103\": container with ID starting with c7fdd81e9718353b611dbdfc27c764a398123bfcc93cfd05d49a58c6159cd103 not found: ID does not exist" Sep 19 05:42:57.857573 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.857550 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77dfff459b-cj4b5"] Sep 19 05:42:57.862184 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:57.862163 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77dfff459b-cj4b5"] Sep 19 05:42:58.266235 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:42:58.266206 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e7ccd71-fe2a-4b18-baec-b17dff3899c2" path="/var/lib/kubelet/pods/5e7ccd71-fe2a-4b18-baec-b17dff3899c2/volumes" Sep 19 05:43:03.761434 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:03.761384 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:43:03.765347 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:03.765323 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6b46b4776c-8zmtc" Sep 19 05:43:07.844499 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:07.844456 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66d6f87bd9-zxf26" podUID="b471a970-9591-4089-92bb-ca69f4d48c91" containerName="console" containerID="cri-o://b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917" gracePeriod=15 Sep 19 05:43:08.094710 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.094641 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:43:08.228180 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.228149 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-service-ca\") pod \"b471a970-9591-4089-92bb-ca69f4d48c91\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " Sep 19 05:43:08.228359 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.228200 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-console-config\") pod \"b471a970-9591-4089-92bb-ca69f4d48c91\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " Sep 19 05:43:08.228359 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.228247 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b471a970-9591-4089-92bb-ca69f4d48c91-console-serving-cert\") pod \"b471a970-9591-4089-92bb-ca69f4d48c91\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " Sep 19 05:43:08.228359 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.228263 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-trusted-ca-bundle\") pod \"b471a970-9591-4089-92bb-ca69f4d48c91\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " Sep 19 05:43:08.228359 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.228297 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-oauth-serving-cert\") pod \"b471a970-9591-4089-92bb-ca69f4d48c91\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " Sep 19 05:43:08.228359 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.228317 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8hb9\" (UniqueName: \"kubernetes.io/projected/b471a970-9591-4089-92bb-ca69f4d48c91-kube-api-access-p8hb9\") pod \"b471a970-9591-4089-92bb-ca69f4d48c91\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " Sep 19 05:43:08.228669 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.228365 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b471a970-9591-4089-92bb-ca69f4d48c91-console-oauth-config\") pod \"b471a970-9591-4089-92bb-ca69f4d48c91\" (UID: \"b471a970-9591-4089-92bb-ca69f4d48c91\") " Sep 19 05:43:08.228669 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.228642 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-service-ca" (OuterVolumeSpecName: "service-ca") pod "b471a970-9591-4089-92bb-ca69f4d48c91" (UID: "b471a970-9591-4089-92bb-ca69f4d48c91"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:43:08.228770 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.228681 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-console-config" (OuterVolumeSpecName: "console-config") pod "b471a970-9591-4089-92bb-ca69f4d48c91" (UID: "b471a970-9591-4089-92bb-ca69f4d48c91"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:43:08.228770 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.228759 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b471a970-9591-4089-92bb-ca69f4d48c91" (UID: "b471a970-9591-4089-92bb-ca69f4d48c91"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:43:08.228897 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.228862 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b471a970-9591-4089-92bb-ca69f4d48c91" (UID: "b471a970-9591-4089-92bb-ca69f4d48c91"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:43:08.230743 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.230710 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b471a970-9591-4089-92bb-ca69f4d48c91-kube-api-access-p8hb9" (OuterVolumeSpecName: "kube-api-access-p8hb9") pod "b471a970-9591-4089-92bb-ca69f4d48c91" (UID: "b471a970-9591-4089-92bb-ca69f4d48c91"). InnerVolumeSpecName "kube-api-access-p8hb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:43:08.230743 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.230726 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b471a970-9591-4089-92bb-ca69f4d48c91-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b471a970-9591-4089-92bb-ca69f4d48c91" (UID: "b471a970-9591-4089-92bb-ca69f4d48c91"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:43:08.230900 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.230761 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b471a970-9591-4089-92bb-ca69f4d48c91-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b471a970-9591-4089-92bb-ca69f4d48c91" (UID: "b471a970-9591-4089-92bb-ca69f4d48c91"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:43:08.329904 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.329874 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b471a970-9591-4089-92bb-ca69f4d48c91-console-oauth-config\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:43:08.329904 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.329902 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:43:08.329904 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.329912 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-console-config\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:43:08.330127 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.329919 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b471a970-9591-4089-92bb-ca69f4d48c91-console-serving-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:43:08.330127 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.329927 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-trusted-ca-bundle\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:43:08.330127 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.329937 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b471a970-9591-4089-92bb-ca69f4d48c91-oauth-serving-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:43:08.330127 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.329945 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8hb9\" (UniqueName: \"kubernetes.io/projected/b471a970-9591-4089-92bb-ca69f4d48c91-kube-api-access-p8hb9\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:43:08.882280 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.882242 2584 generic.go:356] "Generic (PLEG): container finished" podID="b471a970-9591-4089-92bb-ca69f4d48c91" containerID="b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917" exitCode=0 Sep 19 05:43:08.882862 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.882289 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d6f87bd9-zxf26" Sep 19 05:43:08.882862 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.882330 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d6f87bd9-zxf26" event={"ID":"b471a970-9591-4089-92bb-ca69f4d48c91","Type":"ContainerDied","Data":"b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917"} Sep 19 05:43:08.882862 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.882370 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d6f87bd9-zxf26" event={"ID":"b471a970-9591-4089-92bb-ca69f4d48c91","Type":"ContainerDied","Data":"c91ae7465e77625a3ad5d955ad37786380f0adafeaa0e8bf84bea47f227141dd"} Sep 19 05:43:08.882862 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.882394 2584 scope.go:117] "RemoveContainer" containerID="b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917" Sep 19 05:43:08.891651 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.891625 2584 scope.go:117] "RemoveContainer" containerID="b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917" Sep 19 05:43:08.891956 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:43:08.891934 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917\": container with ID starting with b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917 not found: ID does not exist" containerID="b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917" Sep 19 05:43:08.892043 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.891964 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917"} err="failed to get container status \"b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917\": rpc error: code = NotFound desc = could not find container \"b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917\": container with ID starting with b9c8d95eeff04a5bcba93be596cd9ea96023c8408eccd0ace3f586d31346e917 not found: ID does not exist" Sep 19 05:43:08.917610 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.917550 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66d6f87bd9-zxf26"] Sep 19 05:43:08.919317 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:08.919288 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66d6f87bd9-zxf26"] Sep 19 05:43:10.266698 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:10.266665 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b471a970-9591-4089-92bb-ca69f4d48c91" path="/var/lib/kubelet/pods/b471a970-9591-4089-92bb-ca69f4d48c91/volumes" Sep 19 05:43:26.021500 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:26.021456 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:43:26.041020 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:26.040989 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:43:26.977107 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:26.977080 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 05:43:35.483916 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:35.483881 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:43:35.555694 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:35.555664 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:43:37.828414 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.828363 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fdb7ffd46-lgrjz"] Sep 19 05:43:37.828856 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.828756 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9300805f-f719-422b-a428-ae038d9e6a2c" containerName="console" Sep 19 05:43:37.828856 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.828774 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9300805f-f719-422b-a428-ae038d9e6a2c" containerName="console" Sep 19 05:43:37.828856 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.828805 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b471a970-9591-4089-92bb-ca69f4d48c91" containerName="console" Sep 19 05:43:37.828856 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.828811 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b471a970-9591-4089-92bb-ca69f4d48c91" containerName="console" Sep 19 05:43:37.828856 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.828819 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e7ccd71-fe2a-4b18-baec-b17dff3899c2" containerName="console" Sep 19 05:43:37.828856 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.828824 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e7ccd71-fe2a-4b18-baec-b17dff3899c2" containerName="console" Sep 19 05:43:37.829062 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.828879 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b471a970-9591-4089-92bb-ca69f4d48c91" containerName="console" Sep 19 05:43:37.829062 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.828887 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9300805f-f719-422b-a428-ae038d9e6a2c" containerName="console" Sep 19 05:43:37.829062 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.828896 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e7ccd71-fe2a-4b18-baec-b17dff3899c2" containerName="console" Sep 19 05:43:37.831655 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.831635 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.844488 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.844466 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fdb7ffd46-lgrjz"] Sep 19 05:43:37.887969 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.887934 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2zwb\" (UniqueName: \"kubernetes.io/projected/6de8babd-40b4-427f-a097-420cfd6f5d17-kube-api-access-v2zwb\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.888140 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.887979 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6de8babd-40b4-427f-a097-420cfd6f5d17-console-serving-cert\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.888140 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.888000 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-oauth-serving-cert\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.888140 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.888042 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-console-config\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.888140 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.888061 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-trusted-ca-bundle\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.888293 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.888138 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6de8babd-40b4-427f-a097-420cfd6f5d17-console-oauth-config\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.888293 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.888171 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-service-ca\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.903863 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.903833 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:43:37.983006 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.982977 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:43:37.989120 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.989098 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6de8babd-40b4-427f-a097-420cfd6f5d17-console-oauth-config\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.989216 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.989127 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-service-ca\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.989216 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.989159 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2zwb\" (UniqueName: \"kubernetes.io/projected/6de8babd-40b4-427f-a097-420cfd6f5d17-kube-api-access-v2zwb\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.989216 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.989206 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6de8babd-40b4-427f-a097-420cfd6f5d17-console-serving-cert\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.989321 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.989234 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-oauth-serving-cert\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.989468 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.989429 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-console-config\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.989632 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.989498 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-trusted-ca-bundle\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.989977 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.989955 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-service-ca\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.990066 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.989955 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-oauth-serving-cert\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.990147 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.990122 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-console-config\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.990263 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.990249 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-trusted-ca-bundle\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.992243 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.992215 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6de8babd-40b4-427f-a097-420cfd6f5d17-console-oauth-config\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:37.992318 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:37.992299 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6de8babd-40b4-427f-a097-420cfd6f5d17-console-serving-cert\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:38.006634 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:38.006605 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2zwb\" (UniqueName: \"kubernetes.io/projected/6de8babd-40b4-427f-a097-420cfd6f5d17-kube-api-access-v2zwb\") pod \"console-5fdb7ffd46-lgrjz\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:38.140549 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:38.140520 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:38.413988 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:38.413962 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fdb7ffd46-lgrjz"] Sep 19 05:43:38.416504 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:43:38.416472 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6de8babd_40b4_427f_a097_420cfd6f5d17.slice/crio-3f3ab28cbf2e536a80dd88216d4fd264d90236a4df994e0b7b9793843def5fb9 WatchSource:0}: Error finding container 3f3ab28cbf2e536a80dd88216d4fd264d90236a4df994e0b7b9793843def5fb9: Status 404 returned error can't find the container with id 3f3ab28cbf2e536a80dd88216d4fd264d90236a4df994e0b7b9793843def5fb9 Sep 19 05:43:39.014698 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:39.014669 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdb7ffd46-lgrjz" event={"ID":"6de8babd-40b4-427f-a097-420cfd6f5d17","Type":"ContainerStarted","Data":"dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c"} Sep 19 05:43:39.015081 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:39.014714 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdb7ffd46-lgrjz" event={"ID":"6de8babd-40b4-427f-a097-420cfd6f5d17","Type":"ContainerStarted","Data":"3f3ab28cbf2e536a80dd88216d4fd264d90236a4df994e0b7b9793843def5fb9"} Sep 19 05:43:39.031963 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:39.031912 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fdb7ffd46-lgrjz" podStartSLOduration=2.031896105 podStartE2EDuration="2.031896105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:43:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:43:39.030469592 +0000 UTC m=+147.023697122" watchObservedRunningTime="2026-09-19 05:43:39.031896105 +0000 UTC m=+147.025123634" Sep 19 05:43:47.426260 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:47.426218 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:43:47.502554 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:47.502524 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:43:48.141304 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:48.141263 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:48.141520 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:48.141363 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:48.145934 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:48.145910 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:49.061787 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:49.061760 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:43:49.109991 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:49.109962 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55c7f7cf49-k4vjb"] Sep 19 05:43:49.339303 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:49.339274 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:43:49.429191 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:49.429158 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:43:50.830584 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:50.830547 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:43:55.928483 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:55.928390 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:43:56.652764 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:56.652726 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:43:56.753380 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:43:56.753347 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:14.129086 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.129047 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55c7f7cf49-k4vjb" podUID="57c16f60-2e0f-4ab3-a847-01af496daaad" containerName="console" containerID="cri-o://82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9" gracePeriod=15 Sep 19 05:44:14.371820 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.371797 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:44:14.399374 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.399304 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57c16f60-2e0f-4ab3-a847-01af496daaad-console-serving-cert\") pod \"57c16f60-2e0f-4ab3-a847-01af496daaad\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " Sep 19 05:44:14.399374 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.399335 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57c16f60-2e0f-4ab3-a847-01af496daaad-console-oauth-config\") pod \"57c16f60-2e0f-4ab3-a847-01af496daaad\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " Sep 19 05:44:14.399374 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.399364 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-service-ca\") pod \"57c16f60-2e0f-4ab3-a847-01af496daaad\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " Sep 19 05:44:14.399653 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.399392 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-console-config\") pod \"57c16f60-2e0f-4ab3-a847-01af496daaad\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " Sep 19 05:44:14.399653 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.399443 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fg7j\" (UniqueName: \"kubernetes.io/projected/57c16f60-2e0f-4ab3-a847-01af496daaad-kube-api-access-9fg7j\") pod \"57c16f60-2e0f-4ab3-a847-01af496daaad\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " Sep 19 05:44:14.399653 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.399463 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-oauth-serving-cert\") pod \"57c16f60-2e0f-4ab3-a847-01af496daaad\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " Sep 19 05:44:14.399653 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.399480 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-trusted-ca-bundle\") pod \"57c16f60-2e0f-4ab3-a847-01af496daaad\" (UID: \"57c16f60-2e0f-4ab3-a847-01af496daaad\") " Sep 19 05:44:14.399917 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.399889 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-service-ca" (OuterVolumeSpecName: "service-ca") pod "57c16f60-2e0f-4ab3-a847-01af496daaad" (UID: "57c16f60-2e0f-4ab3-a847-01af496daaad"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:44:14.399985 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.399941 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-console-config" (OuterVolumeSpecName: "console-config") pod "57c16f60-2e0f-4ab3-a847-01af496daaad" (UID: "57c16f60-2e0f-4ab3-a847-01af496daaad"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:44:14.399985 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.399962 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "57c16f60-2e0f-4ab3-a847-01af496daaad" (UID: "57c16f60-2e0f-4ab3-a847-01af496daaad"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:44:14.400140 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.400022 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "57c16f60-2e0f-4ab3-a847-01af496daaad" (UID: "57c16f60-2e0f-4ab3-a847-01af496daaad"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:44:14.401635 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.401603 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57c16f60-2e0f-4ab3-a847-01af496daaad-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "57c16f60-2e0f-4ab3-a847-01af496daaad" (UID: "57c16f60-2e0f-4ab3-a847-01af496daaad"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:44:14.401635 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.401627 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57c16f60-2e0f-4ab3-a847-01af496daaad-kube-api-access-9fg7j" (OuterVolumeSpecName: "kube-api-access-9fg7j") pod "57c16f60-2e0f-4ab3-a847-01af496daaad" (UID: "57c16f60-2e0f-4ab3-a847-01af496daaad"). InnerVolumeSpecName "kube-api-access-9fg7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:44:14.401819 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.401797 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57c16f60-2e0f-4ab3-a847-01af496daaad-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "57c16f60-2e0f-4ab3-a847-01af496daaad" (UID: "57c16f60-2e0f-4ab3-a847-01af496daaad"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:44:14.500316 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.500279 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/57c16f60-2e0f-4ab3-a847-01af496daaad-console-serving-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:44:14.500316 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.500311 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/57c16f60-2e0f-4ab3-a847-01af496daaad-console-oauth-config\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:44:14.500316 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.500320 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:44:14.500596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.500329 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-console-config\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:44:14.500596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.500337 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fg7j\" (UniqueName: \"kubernetes.io/projected/57c16f60-2e0f-4ab3-a847-01af496daaad-kube-api-access-9fg7j\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:44:14.500596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.500347 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-oauth-serving-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:44:14.500596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:14.500355 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57c16f60-2e0f-4ab3-a847-01af496daaad-trusted-ca-bundle\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:44:15.165160 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:15.165123 2584 generic.go:356] "Generic (PLEG): container finished" podID="57c16f60-2e0f-4ab3-a847-01af496daaad" containerID="82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9" exitCode=0 Sep 19 05:44:15.165688 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:15.165176 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c7f7cf49-k4vjb" event={"ID":"57c16f60-2e0f-4ab3-a847-01af496daaad","Type":"ContainerDied","Data":"82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9"} Sep 19 05:44:15.165688 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:15.165196 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c7f7cf49-k4vjb" Sep 19 05:44:15.165688 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:15.165241 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c7f7cf49-k4vjb" event={"ID":"57c16f60-2e0f-4ab3-a847-01af496daaad","Type":"ContainerDied","Data":"19070d4e1d60014a537aa6a55c4462b10f3fd3c0af7f7d099ebc42461d1a14c3"} Sep 19 05:44:15.165688 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:15.165263 2584 scope.go:117] "RemoveContainer" containerID="82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9" Sep 19 05:44:15.173727 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:15.173710 2584 scope.go:117] "RemoveContainer" containerID="82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9" Sep 19 05:44:15.174035 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:44:15.174016 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9\": container with ID starting with 82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9 not found: ID does not exist" containerID="82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9" Sep 19 05:44:15.174083 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:15.174044 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9"} err="failed to get container status \"82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9\": rpc error: code = NotFound desc = could not find container \"82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9\": container with ID starting with 82f51d503784a91fb36f6311057d249a283fe452b3e8b6f8c35bcb513342def9 not found: ID does not exist" Sep 19 05:44:15.186866 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:15.186838 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55c7f7cf49-k4vjb"] Sep 19 05:44:15.191994 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:15.191970 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55c7f7cf49-k4vjb"] Sep 19 05:44:16.065837 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:16.065808 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:16.166683 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:16.166650 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:16.266859 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:16.266828 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57c16f60-2e0f-4ab3-a847-01af496daaad" path="/var/lib/kubelet/pods/57c16f60-2e0f-4ab3-a847-01af496daaad/volumes" Sep 19 05:44:27.494654 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:27.494621 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:27.560127 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:27.560086 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:35.163140 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:35.163089 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:38.327348 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:38.327303 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:39.133647 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:39.133599 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:42.127679 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:42.127646 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:42.241987 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:42.241953 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:43.648916 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:43.648887 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:43.764038 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:43.763966 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:44.165443 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:44.165371 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:44.313383 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:44.313347 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:45.260645 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:45.260595 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:46.042530 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:46.042501 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:48.910642 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:48.910612 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:49.011272 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:49.011240 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:51.540203 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:51.540165 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:51.624780 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:51.624744 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:55.261232 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:55.261205 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:56.915765 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:56.915731 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:57.013146 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:57.013112 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:44:58.100607 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.100553 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh"] Sep 19 05:44:58.101013 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.100997 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57c16f60-2e0f-4ab3-a847-01af496daaad" containerName="console" Sep 19 05:44:58.101013 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.101013 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="57c16f60-2e0f-4ab3-a847-01af496daaad" containerName="console" Sep 19 05:44:58.101090 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.101083 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="57c16f60-2e0f-4ab3-a847-01af496daaad" containerName="console" Sep 19 05:44:58.104101 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.104085 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh" Sep 19 05:44:58.107224 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.107202 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 19 05:44:58.107395 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.107245 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-cg56m\"" Sep 19 05:44:58.108060 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.108040 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 19 05:44:58.117666 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.117637 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh"] Sep 19 05:44:58.274419 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.274379 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86q2t\" (UniqueName: \"kubernetes.io/projected/321e0dd2-f379-4880-a307-476a65bb39e5-kube-api-access-86q2t\") pod \"openshift-lws-operator-fd8ccff4c-wh4xh\" (UID: \"321e0dd2-f379-4880-a307-476a65bb39e5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh" Sep 19 05:44:58.274586 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.274459 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/321e0dd2-f379-4880-a307-476a65bb39e5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wh4xh\" (UID: \"321e0dd2-f379-4880-a307-476a65bb39e5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh" Sep 19 05:44:58.375620 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.375524 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86q2t\" (UniqueName: \"kubernetes.io/projected/321e0dd2-f379-4880-a307-476a65bb39e5-kube-api-access-86q2t\") pod \"openshift-lws-operator-fd8ccff4c-wh4xh\" (UID: \"321e0dd2-f379-4880-a307-476a65bb39e5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh" Sep 19 05:44:58.375620 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.375571 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/321e0dd2-f379-4880-a307-476a65bb39e5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wh4xh\" (UID: \"321e0dd2-f379-4880-a307-476a65bb39e5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh" Sep 19 05:44:58.375971 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.375949 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/321e0dd2-f379-4880-a307-476a65bb39e5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wh4xh\" (UID: \"321e0dd2-f379-4880-a307-476a65bb39e5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh" Sep 19 05:44:58.384132 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.384097 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86q2t\" (UniqueName: \"kubernetes.io/projected/321e0dd2-f379-4880-a307-476a65bb39e5-kube-api-access-86q2t\") pod \"openshift-lws-operator-fd8ccff4c-wh4xh\" (UID: \"321e0dd2-f379-4880-a307-476a65bb39e5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh" Sep 19 05:44:58.431720 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.431687 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh" Sep 19 05:44:58.705389 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:58.705180 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh"] Sep 19 05:44:58.708362 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:44:58.708337 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod321e0dd2_f379_4880_a307_476a65bb39e5.slice/crio-6fea1198d0975f465c91586740f044232777b1d04ad990594939cfda084de944 WatchSource:0}: Error finding container 6fea1198d0975f465c91586740f044232777b1d04ad990594939cfda084de944: Status 404 returned error can't find the container with id 6fea1198d0975f465c91586740f044232777b1d04ad990594939cfda084de944 Sep 19 05:44:59.344279 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:44:59.344219 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh" event={"ID":"321e0dd2-f379-4880-a307-476a65bb39e5","Type":"ContainerStarted","Data":"6fea1198d0975f465c91586740f044232777b1d04ad990594939cfda084de944"} Sep 19 05:45:01.354857 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:01.354823 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh" event={"ID":"321e0dd2-f379-4880-a307-476a65bb39e5","Type":"ContainerStarted","Data":"479d446dca62f17d46fe52a9c64e9fd5e2ece0ed89504c1b5254c68c905e2025"} Sep 19 05:45:01.379187 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:01.379130 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wh4xh" podStartSLOduration=1.172139238 podStartE2EDuration="3.379112579s" totalImagesPullingTime="2.206973341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:45:01.377181452 +0000 UTC m=+229.370408983" watchObservedRunningTime="2026-09-19 05:45:01.379112579 +0000 UTC m=+229.372340109" Sep 19 05:45:02.177265 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:02.177230 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:02.284639 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:02.284602 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:03.865168 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:03.865134 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:04.229544 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.229469 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hflj4"] Sep 19 05:45:04.233156 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.233140 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hflj4" Sep 19 05:45:04.236020 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.235999 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-c76vp\"" Sep 19 05:45:04.236199 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.236181 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 19 05:45:04.236423 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.236389 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 19 05:45:04.243952 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.243928 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hflj4"] Sep 19 05:45:04.328985 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.328950 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/561016b8-7b3d-4bca-88d4-7acef09b62fd-bound-sa-token\") pod \"cert-manager-55b4d954d5-hflj4\" (UID: \"561016b8-7b3d-4bca-88d4-7acef09b62fd\") " pod="cert-manager/cert-manager-55b4d954d5-hflj4" Sep 19 05:45:04.329163 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.329013 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrxdl\" (UniqueName: \"kubernetes.io/projected/561016b8-7b3d-4bca-88d4-7acef09b62fd-kube-api-access-mrxdl\") pod \"cert-manager-55b4d954d5-hflj4\" (UID: \"561016b8-7b3d-4bca-88d4-7acef09b62fd\") " pod="cert-manager/cert-manager-55b4d954d5-hflj4" Sep 19 05:45:04.430267 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.430226 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrxdl\" (UniqueName: \"kubernetes.io/projected/561016b8-7b3d-4bca-88d4-7acef09b62fd-kube-api-access-mrxdl\") pod \"cert-manager-55b4d954d5-hflj4\" (UID: \"561016b8-7b3d-4bca-88d4-7acef09b62fd\") " pod="cert-manager/cert-manager-55b4d954d5-hflj4" Sep 19 05:45:04.430517 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.430303 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/561016b8-7b3d-4bca-88d4-7acef09b62fd-bound-sa-token\") pod \"cert-manager-55b4d954d5-hflj4\" (UID: \"561016b8-7b3d-4bca-88d4-7acef09b62fd\") " pod="cert-manager/cert-manager-55b4d954d5-hflj4" Sep 19 05:45:04.438336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.438311 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/561016b8-7b3d-4bca-88d4-7acef09b62fd-bound-sa-token\") pod \"cert-manager-55b4d954d5-hflj4\" (UID: \"561016b8-7b3d-4bca-88d4-7acef09b62fd\") " pod="cert-manager/cert-manager-55b4d954d5-hflj4" Sep 19 05:45:04.438640 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.438620 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrxdl\" (UniqueName: \"kubernetes.io/projected/561016b8-7b3d-4bca-88d4-7acef09b62fd-kube-api-access-mrxdl\") pod \"cert-manager-55b4d954d5-hflj4\" (UID: \"561016b8-7b3d-4bca-88d4-7acef09b62fd\") " pod="cert-manager/cert-manager-55b4d954d5-hflj4" Sep 19 05:45:04.543058 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.542961 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hflj4" Sep 19 05:45:04.837024 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:04.837001 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hflj4"] Sep 19 05:45:04.839092 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:45:04.839051 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod561016b8_7b3d_4bca_88d4_7acef09b62fd.slice/crio-d3688a2072e624815a04b6d4c552c64000d0dd4997e1faaffc678be33d66bc41 WatchSource:0}: Error finding container d3688a2072e624815a04b6d4c552c64000d0dd4997e1faaffc678be33d66bc41: Status 404 returned error can't find the container with id d3688a2072e624815a04b6d4c552c64000d0dd4997e1faaffc678be33d66bc41 Sep 19 05:45:05.375693 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:05.375653 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hflj4" event={"ID":"561016b8-7b3d-4bca-88d4-7acef09b62fd","Type":"ContainerStarted","Data":"d3688a2072e624815a04b6d4c552c64000d0dd4997e1faaffc678be33d66bc41"} Sep 19 05:45:08.391358 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:08.391318 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hflj4" event={"ID":"561016b8-7b3d-4bca-88d4-7acef09b62fd","Type":"ContainerStarted","Data":"c0e8219466f259b19d697d212be82fba352bb685475bc42067cb5a0e157bfc7b"} Sep 19 05:45:08.419718 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:08.419658 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-hflj4" podStartSLOduration=1.60566239 podStartE2EDuration="4.419641985s" totalImagesPullingTime="2.813979595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:45:08.418184428 +0000 UTC m=+236.411411956" watchObservedRunningTime="2026-09-19 05:45:08.419641985 +0000 UTC m=+236.412869514" Sep 19 05:45:12.187329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.187293 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:12.234895 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.234859 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs"] Sep 19 05:45:12.238351 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.238334 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.241722 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.241688 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 19 05:45:12.242688 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.242669 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-g7d5s\"" Sep 19 05:45:12.242861 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.242729 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 19 05:45:12.242861 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.242734 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 19 05:45:12.247886 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.247862 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs"] Sep 19 05:45:12.273624 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.273594 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:12.299964 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.299934 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8whx\" (UniqueName: \"kubernetes.io/projected/34167abd-02cc-4054-9cad-62f55fe837a3-kube-api-access-h8whx\") pod \"lws-controller-manager-58cf5d9b8c-z4jcs\" (UID: \"34167abd-02cc-4054-9cad-62f55fe837a3\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.300083 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.299976 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/34167abd-02cc-4054-9cad-62f55fe837a3-manager-config\") pod \"lws-controller-manager-58cf5d9b8c-z4jcs\" (UID: \"34167abd-02cc-4054-9cad-62f55fe837a3\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.300083 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.299994 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/34167abd-02cc-4054-9cad-62f55fe837a3-metrics-cert\") pod \"lws-controller-manager-58cf5d9b8c-z4jcs\" (UID: \"34167abd-02cc-4054-9cad-62f55fe837a3\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.300083 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.300034 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34167abd-02cc-4054-9cad-62f55fe837a3-cert\") pod \"lws-controller-manager-58cf5d9b8c-z4jcs\" (UID: \"34167abd-02cc-4054-9cad-62f55fe837a3\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.401507 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.401468 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8whx\" (UniqueName: \"kubernetes.io/projected/34167abd-02cc-4054-9cad-62f55fe837a3-kube-api-access-h8whx\") pod \"lws-controller-manager-58cf5d9b8c-z4jcs\" (UID: \"34167abd-02cc-4054-9cad-62f55fe837a3\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.401709 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.401519 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/34167abd-02cc-4054-9cad-62f55fe837a3-manager-config\") pod \"lws-controller-manager-58cf5d9b8c-z4jcs\" (UID: \"34167abd-02cc-4054-9cad-62f55fe837a3\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.401709 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.401637 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/34167abd-02cc-4054-9cad-62f55fe837a3-metrics-cert\") pod \"lws-controller-manager-58cf5d9b8c-z4jcs\" (UID: \"34167abd-02cc-4054-9cad-62f55fe837a3\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.401709 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.401678 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34167abd-02cc-4054-9cad-62f55fe837a3-cert\") pod \"lws-controller-manager-58cf5d9b8c-z4jcs\" (UID: \"34167abd-02cc-4054-9cad-62f55fe837a3\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.402357 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.402337 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/34167abd-02cc-4054-9cad-62f55fe837a3-manager-config\") pod \"lws-controller-manager-58cf5d9b8c-z4jcs\" (UID: \"34167abd-02cc-4054-9cad-62f55fe837a3\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.404181 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.404161 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/34167abd-02cc-4054-9cad-62f55fe837a3-metrics-cert\") pod \"lws-controller-manager-58cf5d9b8c-z4jcs\" (UID: \"34167abd-02cc-4054-9cad-62f55fe837a3\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.404305 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.404286 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34167abd-02cc-4054-9cad-62f55fe837a3-cert\") pod \"lws-controller-manager-58cf5d9b8c-z4jcs\" (UID: \"34167abd-02cc-4054-9cad-62f55fe837a3\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.410655 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.410636 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8whx\" (UniqueName: \"kubernetes.io/projected/34167abd-02cc-4054-9cad-62f55fe837a3-kube-api-access-h8whx\") pod \"lws-controller-manager-58cf5d9b8c-z4jcs\" (UID: \"34167abd-02cc-4054-9cad-62f55fe837a3\") " pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.548496 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.548407 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:12.822960 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:12.822933 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs"] Sep 19 05:45:12.825217 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:45:12.825189 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34167abd_02cc_4054_9cad_62f55fe837a3.slice/crio-17eade035db5f477bba8574ebff40d2faa02427dbca5c6b27b99c164d36d57e4 WatchSource:0}: Error finding container 17eade035db5f477bba8574ebff40d2faa02427dbca5c6b27b99c164d36d57e4: Status 404 returned error can't find the container with id 17eade035db5f477bba8574ebff40d2faa02427dbca5c6b27b99c164d36d57e4 Sep 19 05:45:13.413231 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:13.413191 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" event={"ID":"34167abd-02cc-4054-9cad-62f55fe837a3","Type":"ContainerStarted","Data":"17eade035db5f477bba8574ebff40d2faa02427dbca5c6b27b99c164d36d57e4"} Sep 19 05:45:14.418486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:14.418454 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" event={"ID":"34167abd-02cc-4054-9cad-62f55fe837a3","Type":"ContainerStarted","Data":"e2319a29d712dc8fd4adad1a34a2c72e98295ff3fb766446b9de37686d92cd36"} Sep 19 05:45:14.418911 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:14.418581 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:14.438346 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:14.438297 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" podStartSLOduration=1.218826571 podStartE2EDuration="2.438276541s" totalImagesPullingTime="1.21944997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:45:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:45:14.437260827 +0000 UTC m=+242.430488362" watchObservedRunningTime="2026-09-19 05:45:14.438276541 +0000 UTC m=+242.431504072" Sep 19 05:45:20.072993 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:20.072955 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:20.210341 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:20.210305 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:20.681148 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:20.681119 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:21.060457 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.060378 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:21.171317 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.171284 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:21.260428 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.260384 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j"] Sep 19 05:45:21.265304 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.265283 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.268129 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.268083 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 19 05:45:21.268276 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.268236 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 19 05:45:21.268276 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.268251 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 19 05:45:21.268365 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.268251 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-s27qb\"" Sep 19 05:45:21.268547 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.268533 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 19 05:45:21.304774 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.304732 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j"] Sep 19 05:45:21.386132 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.386095 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/732f2a56-951f-4a7b-a3b7-2be38ff1c869-webhook-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.386315 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.386145 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/732f2a56-951f-4a7b-a3b7-2be38ff1c869-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.386315 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.386193 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/732f2a56-951f-4a7b-a3b7-2be38ff1c869-manifests\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.386315 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.386307 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/732f2a56-951f-4a7b-a3b7-2be38ff1c869-tmp\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.386473 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.386360 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2fdk\" (UniqueName: \"kubernetes.io/projected/732f2a56-951f-4a7b-a3b7-2be38ff1c869-kube-api-access-p2fdk\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.448555 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.448516 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:21.487323 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.487290 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2fdk\" (UniqueName: \"kubernetes.io/projected/732f2a56-951f-4a7b-a3b7-2be38ff1c869-kube-api-access-p2fdk\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.487520 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.487344 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/732f2a56-951f-4a7b-a3b7-2be38ff1c869-webhook-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.487520 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.487370 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/732f2a56-951f-4a7b-a3b7-2be38ff1c869-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.487520 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.487428 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/732f2a56-951f-4a7b-a3b7-2be38ff1c869-manifests\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.487520 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.487510 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/732f2a56-951f-4a7b-a3b7-2be38ff1c869-tmp\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.487964 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.487940 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/732f2a56-951f-4a7b-a3b7-2be38ff1c869-manifests\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.489873 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.489850 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/732f2a56-951f-4a7b-a3b7-2be38ff1c869-tmp\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.490176 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.490155 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/732f2a56-951f-4a7b-a3b7-2be38ff1c869-webhook-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.490241 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.490164 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/732f2a56-951f-4a7b-a3b7-2be38ff1c869-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.495841 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.495819 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2fdk\" (UniqueName: \"kubernetes.io/projected/732f2a56-951f-4a7b-a3b7-2be38ff1c869-kube-api-access-p2fdk\") pod \"opendatahub-operator-controller-manager-fd6794c5b-ht29j\" (UID: \"732f2a56-951f-4a7b-a3b7-2be38ff1c869\") " pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.547384 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.547347 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:21.576425 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.576386 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:21.686726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.686625 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:21.789463 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.789432 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:21.855316 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.855292 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j"] Sep 19 05:45:21.857995 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:45:21.857973 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod732f2a56_951f_4a7b_a3b7_2be38ff1c869.slice/crio-cd4a6ad75474fc7d493cb2f487753e95a2614585d146c7c37be445a8a10d3578 WatchSource:0}: Error finding container cd4a6ad75474fc7d493cb2f487753e95a2614585d146c7c37be445a8a10d3578: Status 404 returned error can't find the container with id cd4a6ad75474fc7d493cb2f487753e95a2614585d146c7c37be445a8a10d3578 Sep 19 05:45:21.903771 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:21.903742 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:22.006690 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:22.006606 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:22.123850 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:22.123820 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:22.461141 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:22.461099 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" event={"ID":"732f2a56-951f-4a7b-a3b7-2be38ff1c869","Type":"ContainerStarted","Data":"cd4a6ad75474fc7d493cb2f487753e95a2614585d146c7c37be445a8a10d3578"} Sep 19 05:45:25.425166 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:25.425089 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58cf5d9b8c-z4jcs" Sep 19 05:45:25.479731 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:25.479695 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" event={"ID":"732f2a56-951f-4a7b-a3b7-2be38ff1c869","Type":"ContainerStarted","Data":"8614288e7e712d5c27f6ef3fdfb7253e72d160b37e432aacebf9d20e663e4236"} Sep 19 05:45:26.485919 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:26.485886 2584 generic.go:356] "Generic (PLEG): container finished" podID="732f2a56-951f-4a7b-a3b7-2be38ff1c869" containerID="8614288e7e712d5c27f6ef3fdfb7253e72d160b37e432aacebf9d20e663e4236" exitCode=0 Sep 19 05:45:26.486283 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:26.485972 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" event={"ID":"732f2a56-951f-4a7b-a3b7-2be38ff1c869","Type":"ContainerDied","Data":"8614288e7e712d5c27f6ef3fdfb7253e72d160b37e432aacebf9d20e663e4236"} Sep 19 05:45:27.493279 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:27.493240 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" event={"ID":"732f2a56-951f-4a7b-a3b7-2be38ff1c869","Type":"ContainerStarted","Data":"2000f6c0bfb4b989fa3dc1e7a67733ca9197276736c95a02ae31eb721b4a4618"} Sep 19 05:45:27.493702 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:27.493424 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:27.529493 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:27.529442 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" podStartSLOduration=2.045369052 podStartE2EDuration="6.529426914s" totalImagesPullingTime="3.398648238s" totalInitContainerRuntime="1.085409624s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:45:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:45:27.527929379 +0000 UTC m=+255.521156910" watchObservedRunningTime="2026-09-19 05:45:27.529426914 +0000 UTC m=+255.522654442" Sep 19 05:45:38.500264 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:38.500233 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fd6794c5b-ht29j" Sep 19 05:45:41.907513 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:41.907479 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:42.031092 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:42.031049 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:44.648090 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:44.648057 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:44.750906 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:44.750874 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:47.700493 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.700453 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:47.741661 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.741632 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg"] Sep 19 05:45:47.756652 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.756629 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" Sep 19 05:45:47.760691 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.760664 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg"] Sep 19 05:45:47.760832 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.760716 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 19 05:45:47.760832 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.760804 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 19 05:45:47.761121 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.761102 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 19 05:45:47.761121 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.761116 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 19 05:45:47.761220 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.761126 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-lfvvb\"" Sep 19 05:45:47.825831 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.825793 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e631cb4-6f0e-4c0f-9350-8aaff40a84f9-tmp\") pod \"kube-auth-proxy-6b984b98c7-r7tgg\" (UID: \"6e631cb4-6f0e-4c0f-9350-8aaff40a84f9\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" Sep 19 05:45:47.826032 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.825851 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e631cb4-6f0e-4c0f-9350-8aaff40a84f9-tls-certs\") pod \"kube-auth-proxy-6b984b98c7-r7tgg\" (UID: \"6e631cb4-6f0e-4c0f-9350-8aaff40a84f9\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" Sep 19 05:45:47.826032 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.825941 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkqnd\" (UniqueName: \"kubernetes.io/projected/6e631cb4-6f0e-4c0f-9350-8aaff40a84f9-kube-api-access-rkqnd\") pod \"kube-auth-proxy-6b984b98c7-r7tgg\" (UID: \"6e631cb4-6f0e-4c0f-9350-8aaff40a84f9\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" Sep 19 05:45:47.926900 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.926860 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkqnd\" (UniqueName: \"kubernetes.io/projected/6e631cb4-6f0e-4c0f-9350-8aaff40a84f9-kube-api-access-rkqnd\") pod \"kube-auth-proxy-6b984b98c7-r7tgg\" (UID: \"6e631cb4-6f0e-4c0f-9350-8aaff40a84f9\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" Sep 19 05:45:47.927086 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.926924 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e631cb4-6f0e-4c0f-9350-8aaff40a84f9-tmp\") pod \"kube-auth-proxy-6b984b98c7-r7tgg\" (UID: \"6e631cb4-6f0e-4c0f-9350-8aaff40a84f9\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" Sep 19 05:45:47.927086 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.926966 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e631cb4-6f0e-4c0f-9350-8aaff40a84f9-tls-certs\") pod \"kube-auth-proxy-6b984b98c7-r7tgg\" (UID: \"6e631cb4-6f0e-4c0f-9350-8aaff40a84f9\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" Sep 19 05:45:47.927086 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:45:47.927079 2584 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 19 05:45:47.927207 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:45:47.927140 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e631cb4-6f0e-4c0f-9350-8aaff40a84f9-tls-certs podName:6e631cb4-6f0e-4c0f-9350-8aaff40a84f9 nodeName:}" failed. No retries permitted until 2026-09-19 05:45:48.42712515 +0000 UTC m=+276.420352663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6e631cb4-6f0e-4c0f-9350-8aaff40a84f9-tls-certs") pod "kube-auth-proxy-6b984b98c7-r7tgg" (UID: "6e631cb4-6f0e-4c0f-9350-8aaff40a84f9") : secret "kube-auth-proxy-tls" not found Sep 19 05:45:47.929260 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.929240 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e631cb4-6f0e-4c0f-9350-8aaff40a84f9-tmp\") pod \"kube-auth-proxy-6b984b98c7-r7tgg\" (UID: \"6e631cb4-6f0e-4c0f-9350-8aaff40a84f9\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" Sep 19 05:45:47.936154 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:47.936127 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkqnd\" (UniqueName: \"kubernetes.io/projected/6e631cb4-6f0e-4c0f-9350-8aaff40a84f9-kube-api-access-rkqnd\") pod \"kube-auth-proxy-6b984b98c7-r7tgg\" (UID: \"6e631cb4-6f0e-4c0f-9350-8aaff40a84f9\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" Sep 19 05:45:48.216155 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:48.216123 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:48.432090 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:48.432047 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e631cb4-6f0e-4c0f-9350-8aaff40a84f9-tls-certs\") pod \"kube-auth-proxy-6b984b98c7-r7tgg\" (UID: \"6e631cb4-6f0e-4c0f-9350-8aaff40a84f9\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" Sep 19 05:45:48.434456 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:48.434429 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e631cb4-6f0e-4c0f-9350-8aaff40a84f9-tls-certs\") pod \"kube-auth-proxy-6b984b98c7-r7tgg\" (UID: \"6e631cb4-6f0e-4c0f-9350-8aaff40a84f9\") " pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" Sep 19 05:45:48.667630 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:48.667589 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" Sep 19 05:45:48.948948 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:48.948923 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg"] Sep 19 05:45:48.951506 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:45:48.951460 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e631cb4_6f0e_4c0f_9350_8aaff40a84f9.slice/crio-a2e85ecc37ec5b119440d8f6a124817ca7223cbb32f24b575026bdbcbe8c357d WatchSource:0}: Error finding container a2e85ecc37ec5b119440d8f6a124817ca7223cbb32f24b575026bdbcbe8c357d: Status 404 returned error can't find the container with id a2e85ecc37ec5b119440d8f6a124817ca7223cbb32f24b575026bdbcbe8c357d Sep 19 05:45:49.578771 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:49.578732 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:49.596945 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:49.596908 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" event={"ID":"6e631cb4-6f0e-4c0f-9350-8aaff40a84f9","Type":"ContainerStarted","Data":"a2e85ecc37ec5b119440d8f6a124817ca7223cbb32f24b575026bdbcbe8c357d"} Sep 19 05:45:49.711670 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:49.711633 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:50.832178 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:50.832138 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:52.612308 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:52.612272 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" event={"ID":"6e631cb4-6f0e-4c0f-9350-8aaff40a84f9","Type":"ContainerStarted","Data":"8d955d433e5cab0b7e9d47222a93cef494c67e183f062d85d142ec77119af6c4"} Sep 19 05:45:53.324800 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:53.324765 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:53.432500 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:53.432464 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:53.524932 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:53.524902 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:53.629822 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:53.629793 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:54.069298 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:54.069204 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:54.224636 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:54.224604 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:54.714716 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:54.714683 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:54.819962 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:54.819927 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:55.765234 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:55.765196 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:56.094720 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:56.094679 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:45:56.493256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:45:56.493172 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:46:04.188455 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:46:04.188422 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:46:05.464272 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:46:05.464240 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:46:14.136124 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:46:14.136095 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 05:46:14.141667 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:46:14.141641 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 05:46:14.143071 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:46:14.143051 2584 kubelet.go:1648] "Image garbage collection succeeded" Sep 19 05:46:53.462478 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:46:53.462437 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:46:53.563028 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:46:53.563001 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:46:54.165719 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:46:54.165687 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:46:54.267849 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:46:54.267824 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:10.223882 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:10.223795 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:12.600421 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:12.600361 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:12.980021 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:12.979930 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:14.373610 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.373546 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6b984b98c7-r7tgg" podStartSLOduration=84.695554167 podStartE2EDuration="1m27.373527827s" totalImagesPullingTime="2.67797366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:45:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:45:52.640137177 +0000 UTC m=+280.633364708" watchObservedRunningTime="2026-09-19 05:47:14.373527827 +0000 UTC m=+362.366755358" Sep 19 05:47:14.375374 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.375346 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bc6d7896d-j64jf"] Sep 19 05:47:14.378785 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.378767 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.393357 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.393335 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bc6d7896d-j64jf"] Sep 19 05:47:14.446605 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.446572 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:14.517815 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.517771 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-console-serving-cert\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.517815 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.517816 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-service-ca\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.518075 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.517858 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-oauth-serving-cert\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.518075 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.517881 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq756\" (UniqueName: \"kubernetes.io/projected/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-kube-api-access-gq756\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.518075 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.517921 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-console-oauth-config\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.518075 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.517947 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-trusted-ca-bundle\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.518231 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.518076 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-console-config\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.552174 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.552146 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:14.618663 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.618620 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-console-config\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.618855 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.618684 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-console-serving-cert\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.618855 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.618707 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-service-ca\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.618855 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.618735 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-oauth-serving-cert\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.618855 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.618756 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq756\" (UniqueName: \"kubernetes.io/projected/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-kube-api-access-gq756\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.618855 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.618791 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-console-oauth-config\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.618855 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.618812 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-trusted-ca-bundle\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.619588 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.619539 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-console-config\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.619588 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.619561 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-service-ca\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.619588 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.619561 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-oauth-serving-cert\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.619777 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.619692 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-trusted-ca-bundle\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.621258 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.621240 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-console-serving-cert\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.621341 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.621305 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-console-oauth-config\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.629241 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.629190 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq756\" (UniqueName: \"kubernetes.io/projected/ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83-kube-api-access-gq756\") pod \"console-5bc6d7896d-j64jf\" (UID: \"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83\") " pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.650258 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.650232 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:14.687842 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.687815 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:14.965753 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:14.965727 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bc6d7896d-j64jf"] Sep 19 05:47:14.968157 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:47:14.968131 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7d33d3_a4dd_4a1b_bedc_34bfb6298d83.slice/crio-1adc6c63797e9100654d4f2d4027cbc6e20c8baf931d7e75cf81f4db996ead37 WatchSource:0}: Error finding container 1adc6c63797e9100654d4f2d4027cbc6e20c8baf931d7e75cf81f4db996ead37: Status 404 returned error can't find the container with id 1adc6c63797e9100654d4f2d4027cbc6e20c8baf931d7e75cf81f4db996ead37 Sep 19 05:47:15.002588 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:15.002558 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bc6d7896d-j64jf" event={"ID":"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83","Type":"ContainerStarted","Data":"1adc6c63797e9100654d4f2d4027cbc6e20c8baf931d7e75cf81f4db996ead37"} Sep 19 05:47:15.209615 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:15.209581 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:15.391694 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:15.391659 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:15.545461 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:15.545427 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:16.009165 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:16.009130 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bc6d7896d-j64jf" event={"ID":"ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83","Type":"ContainerStarted","Data":"9a85e46693f38a6186ec19378a52fa57015a47a1a3d76a156eff1949cdcad108"} Sep 19 05:47:16.030776 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:16.030724 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bc6d7896d-j64jf" podStartSLOduration=2.030704927 podStartE2EDuration="2.030704927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:47:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:47:16.029283875 +0000 UTC m=+364.022511410" watchObservedRunningTime="2026-09-19 05:47:16.030704927 +0000 UTC m=+364.023932458" Sep 19 05:47:16.676725 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:16.676694 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:17.712723 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:17.712688 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:17.813162 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:17.813132 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:18.401891 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:18.401851 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:18.638789 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:18.638751 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:18.810325 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:18.810206 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:18.952035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:18.951969 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:23.807889 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:23.807854 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:23.911349 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:23.911319 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:24.688946 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:24.688912 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:24.689134 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:24.689049 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:24.693854 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:24.693827 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:25.056164 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:25.056077 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bc6d7896d-j64jf" Sep 19 05:47:25.115727 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:25.115693 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fdb7ffd46-lgrjz"] Sep 19 05:47:25.263933 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:25.263903 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:25.365975 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:25.365928 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:25.466212 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:25.466179 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:27.445720 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:27.445655 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:27.561793 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:27.561759 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:28.616468 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:28.616433 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:29.086986 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:29.086954 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:29.294806 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:29.294771 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:29.920675 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:29.920645 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:30.017811 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:30.017783 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:36.431736 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:36.431702 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:36.562440 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:36.562383 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:36.746499 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:36.746382 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:43.316515 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:43.316482 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:43.429656 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:43.429623 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:46.883327 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:46.883285 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:46.982797 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:46.982756 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:47.105758 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:47.105721 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:47.217118 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:47.217026 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:50.137061 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.137022 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5fdb7ffd46-lgrjz" podUID="6de8babd-40b4-427f-a097-420cfd6f5d17" containerName="console" containerID="cri-o://dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c" gracePeriod=15 Sep 19 05:47:50.379327 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.379296 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:47:50.550195 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.550102 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-trusted-ca-bundle\") pod \"6de8babd-40b4-427f-a097-420cfd6f5d17\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " Sep 19 05:47:50.550195 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.550153 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6de8babd-40b4-427f-a097-420cfd6f5d17-console-oauth-config\") pod \"6de8babd-40b4-427f-a097-420cfd6f5d17\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " Sep 19 05:47:50.550195 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.550185 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-oauth-serving-cert\") pod \"6de8babd-40b4-427f-a097-420cfd6f5d17\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " Sep 19 05:47:50.550492 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.550235 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6de8babd-40b4-427f-a097-420cfd6f5d17-console-serving-cert\") pod \"6de8babd-40b4-427f-a097-420cfd6f5d17\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " Sep 19 05:47:50.550492 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.550296 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-console-config\") pod \"6de8babd-40b4-427f-a097-420cfd6f5d17\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " Sep 19 05:47:50.550492 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.550348 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-service-ca\") pod \"6de8babd-40b4-427f-a097-420cfd6f5d17\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " Sep 19 05:47:50.550492 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.550383 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2zwb\" (UniqueName: \"kubernetes.io/projected/6de8babd-40b4-427f-a097-420cfd6f5d17-kube-api-access-v2zwb\") pod \"6de8babd-40b4-427f-a097-420cfd6f5d17\" (UID: \"6de8babd-40b4-427f-a097-420cfd6f5d17\") " Sep 19 05:47:50.550740 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.550710 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6de8babd-40b4-427f-a097-420cfd6f5d17" (UID: "6de8babd-40b4-427f-a097-420cfd6f5d17"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:47:50.550795 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.550730 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-service-ca" (OuterVolumeSpecName: "service-ca") pod "6de8babd-40b4-427f-a097-420cfd6f5d17" (UID: "6de8babd-40b4-427f-a097-420cfd6f5d17"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:47:50.550795 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.550745 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-console-config" (OuterVolumeSpecName: "console-config") pod "6de8babd-40b4-427f-a097-420cfd6f5d17" (UID: "6de8babd-40b4-427f-a097-420cfd6f5d17"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:47:50.550976 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.550958 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6de8babd-40b4-427f-a097-420cfd6f5d17" (UID: "6de8babd-40b4-427f-a097-420cfd6f5d17"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:47:50.552456 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.552424 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6de8babd-40b4-427f-a097-420cfd6f5d17-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6de8babd-40b4-427f-a097-420cfd6f5d17" (UID: "6de8babd-40b4-427f-a097-420cfd6f5d17"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:47:50.552601 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.552582 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6de8babd-40b4-427f-a097-420cfd6f5d17-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6de8babd-40b4-427f-a097-420cfd6f5d17" (UID: "6de8babd-40b4-427f-a097-420cfd6f5d17"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:47:50.552649 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.552584 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6de8babd-40b4-427f-a097-420cfd6f5d17-kube-api-access-v2zwb" (OuterVolumeSpecName: "kube-api-access-v2zwb") pod "6de8babd-40b4-427f-a097-420cfd6f5d17" (UID: "6de8babd-40b4-427f-a097-420cfd6f5d17"). InnerVolumeSpecName "kube-api-access-v2zwb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:47:50.651927 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.651892 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6de8babd-40b4-427f-a097-420cfd6f5d17-console-oauth-config\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:47:50.651927 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.651922 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-oauth-serving-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:47:50.651927 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.651932 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6de8babd-40b4-427f-a097-420cfd6f5d17-console-serving-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:47:50.652155 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.651942 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-console-config\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:47:50.652155 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.651950 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:47:50.652155 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.651958 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2zwb\" (UniqueName: \"kubernetes.io/projected/6de8babd-40b4-427f-a097-420cfd6f5d17-kube-api-access-v2zwb\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:47:50.652155 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.651967 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6de8babd-40b4-427f-a097-420cfd6f5d17-trusted-ca-bundle\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:47:50.685459 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.685425 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:50.759297 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:50.759270 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:51.177124 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:51.177038 2584 generic.go:356] "Generic (PLEG): container finished" podID="6de8babd-40b4-427f-a097-420cfd6f5d17" containerID="dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c" exitCode=0 Sep 19 05:47:51.177124 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:51.177082 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdb7ffd46-lgrjz" event={"ID":"6de8babd-40b4-427f-a097-420cfd6f5d17","Type":"ContainerDied","Data":"dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c"} Sep 19 05:47:51.177124 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:51.177087 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fdb7ffd46-lgrjz" Sep 19 05:47:51.177124 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:51.177121 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fdb7ffd46-lgrjz" event={"ID":"6de8babd-40b4-427f-a097-420cfd6f5d17","Type":"ContainerDied","Data":"3f3ab28cbf2e536a80dd88216d4fd264d90236a4df994e0b7b9793843def5fb9"} Sep 19 05:47:51.177708 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:51.177141 2584 scope.go:117] "RemoveContainer" containerID="dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c" Sep 19 05:47:51.186099 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:51.186082 2584 scope.go:117] "RemoveContainer" containerID="dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c" Sep 19 05:47:51.186418 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:47:51.186383 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c\": container with ID starting with dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c not found: ID does not exist" containerID="dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c" Sep 19 05:47:51.186486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:51.186432 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c"} err="failed to get container status \"dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c\": rpc error: code = NotFound desc = could not find container \"dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c\": container with ID starting with dc504832e823857a2522849cd772bb347599d646fa377a160d21a400c5ca6c0c not found: ID does not exist" Sep 19 05:47:51.200737 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:51.200700 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fdb7ffd46-lgrjz"] Sep 19 05:47:51.202339 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:51.202318 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fdb7ffd46-lgrjz"] Sep 19 05:47:52.266455 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:52.266414 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6de8babd-40b4-427f-a097-420cfd6f5d17" path="/var/lib/kubelet/pods/6de8babd-40b4-427f-a097-420cfd6f5d17/volumes" Sep 19 05:47:53.787809 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:53.787778 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:53.857829 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:53.857799 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:56.506734 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:56.506697 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:56.637246 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:56.637216 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:47:56.726966 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:47:56.726937 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:10.732764 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:10.732726 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:10.977059 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:10.977029 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:12.985726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:12.985683 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-l25xc"] Sep 19 05:48:12.986813 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:12.986781 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6de8babd-40b4-427f-a097-420cfd6f5d17" containerName="console" Sep 19 05:48:12.986970 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:12.986957 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6de8babd-40b4-427f-a097-420cfd6f5d17" containerName="console" Sep 19 05:48:12.987190 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:12.987175 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6de8babd-40b4-427f-a097-420cfd6f5d17" containerName="console" Sep 19 05:48:12.990608 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:12.990590 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-l25xc" Sep 19 05:48:12.995772 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:12.995751 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jgws2\"" Sep 19 05:48:12.996344 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:12.996046 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 19 05:48:12.996645 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:12.996095 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 19 05:48:12.999325 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:12.997369 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 19 05:48:13.000464 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.000447 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-l25xc"] Sep 19 05:48:13.003894 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.003859 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:13.058799 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.058768 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6m9r\" (UniqueName: \"kubernetes.io/projected/ba0ab1a6-6439-4a99-976f-cd2aae40b137-kube-api-access-b6m9r\") pod \"limitador-limitador-8588749977-l25xc\" (UID: \"ba0ab1a6-6439-4a99-976f-cd2aae40b137\") " pod="kuadrant-system/limitador-limitador-8588749977-l25xc" Sep 19 05:48:13.058961 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.058816 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ba0ab1a6-6439-4a99-976f-cd2aae40b137-config-file\") pod \"limitador-limitador-8588749977-l25xc\" (UID: \"ba0ab1a6-6439-4a99-976f-cd2aae40b137\") " pod="kuadrant-system/limitador-limitador-8588749977-l25xc" Sep 19 05:48:13.075200 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.075165 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-l25xc"] Sep 19 05:48:13.094853 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.094816 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:13.159721 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.159682 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6m9r\" (UniqueName: \"kubernetes.io/projected/ba0ab1a6-6439-4a99-976f-cd2aae40b137-kube-api-access-b6m9r\") pod \"limitador-limitador-8588749977-l25xc\" (UID: \"ba0ab1a6-6439-4a99-976f-cd2aae40b137\") " pod="kuadrant-system/limitador-limitador-8588749977-l25xc" Sep 19 05:48:13.159892 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.159738 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ba0ab1a6-6439-4a99-976f-cd2aae40b137-config-file\") pod \"limitador-limitador-8588749977-l25xc\" (UID: \"ba0ab1a6-6439-4a99-976f-cd2aae40b137\") " pod="kuadrant-system/limitador-limitador-8588749977-l25xc" Sep 19 05:48:13.160384 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.160361 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ba0ab1a6-6439-4a99-976f-cd2aae40b137-config-file\") pod \"limitador-limitador-8588749977-l25xc\" (UID: \"ba0ab1a6-6439-4a99-976f-cd2aae40b137\") " pod="kuadrant-system/limitador-limitador-8588749977-l25xc" Sep 19 05:48:13.168837 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.168810 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6m9r\" (UniqueName: \"kubernetes.io/projected/ba0ab1a6-6439-4a99-976f-cd2aae40b137-kube-api-access-b6m9r\") pod \"limitador-limitador-8588749977-l25xc\" (UID: \"ba0ab1a6-6439-4a99-976f-cd2aae40b137\") " pod="kuadrant-system/limitador-limitador-8588749977-l25xc" Sep 19 05:48:13.308124 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.308053 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-l25xc" Sep 19 05:48:13.580786 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.580758 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-l25xc"] Sep 19 05:48:13.583005 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:48:13.582978 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba0ab1a6_6439_4a99_976f_cd2aae40b137.slice/crio-f31b781f110fd4b523db29c61304ebb7b996286bd5a3f715c5279382bce45c9c WatchSource:0}: Error finding container f31b781f110fd4b523db29c61304ebb7b996286bd5a3f715c5279382bce45c9c: Status 404 returned error can't find the container with id f31b781f110fd4b523db29c61304ebb7b996286bd5a3f715c5279382bce45c9c Sep 19 05:48:13.585096 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.585080 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 05:48:13.750662 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.750624 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lmhw7"] Sep 19 05:48:13.754966 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.754949 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" Sep 19 05:48:13.757475 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.757443 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vknd7\"" Sep 19 05:48:13.761375 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.761348 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lmhw7"] Sep 19 05:48:13.793824 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.793795 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:13.867135 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.867038 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnnfk\" (UniqueName: \"kubernetes.io/projected/49a6b7a0-a424-4d6d-9875-4f8ddadc949e-kube-api-access-xnnfk\") pod \"authorino-7bcfd8f9cd-lmhw7\" (UID: \"49a6b7a0-a424-4d6d-9875-4f8ddadc949e\") " pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" Sep 19 05:48:13.894203 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.894173 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:13.896806 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.896781 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-bn499"] Sep 19 05:48:13.900009 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.899994 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-bn499" Sep 19 05:48:13.916649 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.916624 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-bn499"] Sep 19 05:48:13.967774 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.967734 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnnfk\" (UniqueName: \"kubernetes.io/projected/49a6b7a0-a424-4d6d-9875-4f8ddadc949e-kube-api-access-xnnfk\") pod \"authorino-7bcfd8f9cd-lmhw7\" (UID: \"49a6b7a0-a424-4d6d-9875-4f8ddadc949e\") " pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" Sep 19 05:48:13.967960 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.967807 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wv4z\" (UniqueName: \"kubernetes.io/projected/369027fc-839c-4973-8009-6c144a7aa52e-kube-api-access-2wv4z\") pod \"authorino-59c974bd48-bn499\" (UID: \"369027fc-839c-4973-8009-6c144a7aa52e\") " pod="kuadrant-system/authorino-59c974bd48-bn499" Sep 19 05:48:13.977724 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:13.977698 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnnfk\" (UniqueName: \"kubernetes.io/projected/49a6b7a0-a424-4d6d-9875-4f8ddadc949e-kube-api-access-xnnfk\") pod \"authorino-7bcfd8f9cd-lmhw7\" (UID: \"49a6b7a0-a424-4d6d-9875-4f8ddadc949e\") " pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" Sep 19 05:48:14.066644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:14.066604 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" Sep 19 05:48:14.069057 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:14.069029 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wv4z\" (UniqueName: \"kubernetes.io/projected/369027fc-839c-4973-8009-6c144a7aa52e-kube-api-access-2wv4z\") pod \"authorino-59c974bd48-bn499\" (UID: \"369027fc-839c-4973-8009-6c144a7aa52e\") " pod="kuadrant-system/authorino-59c974bd48-bn499" Sep 19 05:48:14.078168 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:14.078141 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wv4z\" (UniqueName: \"kubernetes.io/projected/369027fc-839c-4973-8009-6c144a7aa52e-kube-api-access-2wv4z\") pod \"authorino-59c974bd48-bn499\" (UID: \"369027fc-839c-4973-8009-6c144a7aa52e\") " pod="kuadrant-system/authorino-59c974bd48-bn499" Sep 19 05:48:14.209317 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:14.209284 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-bn499" Sep 19 05:48:14.287553 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:14.287523 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-l25xc" event={"ID":"ba0ab1a6-6439-4a99-976f-cd2aae40b137","Type":"ContainerStarted","Data":"f31b781f110fd4b523db29c61304ebb7b996286bd5a3f715c5279382bce45c9c"} Sep 19 05:48:14.339784 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:14.339764 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lmhw7"] Sep 19 05:48:14.342229 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:48:14.342203 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49a6b7a0_a424_4d6d_9875_4f8ddadc949e.slice/crio-3c8a67d6d2035e60fe3b934bacad9ab5d6cd7d3f582b92bf1d2ccc25140ea53b WatchSource:0}: Error finding container 3c8a67d6d2035e60fe3b934bacad9ab5d6cd7d3f582b92bf1d2ccc25140ea53b: Status 404 returned error can't find the container with id 3c8a67d6d2035e60fe3b934bacad9ab5d6cd7d3f582b92bf1d2ccc25140ea53b Sep 19 05:48:14.478993 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:14.478961 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-bn499"] Sep 19 05:48:14.481893 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:48:14.481858 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod369027fc_839c_4973_8009_6c144a7aa52e.slice/crio-f788cf77e7195aadc7058931b559278f77f304732c3b5664f84e45cda4c8b795 WatchSource:0}: Error finding container f788cf77e7195aadc7058931b559278f77f304732c3b5664f84e45cda4c8b795: Status 404 returned error can't find the container with id f788cf77e7195aadc7058931b559278f77f304732c3b5664f84e45cda4c8b795 Sep 19 05:48:15.295324 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:15.295250 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" event={"ID":"49a6b7a0-a424-4d6d-9875-4f8ddadc949e","Type":"ContainerStarted","Data":"3c8a67d6d2035e60fe3b934bacad9ab5d6cd7d3f582b92bf1d2ccc25140ea53b"} Sep 19 05:48:15.297667 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:15.297634 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-bn499" event={"ID":"369027fc-839c-4973-8009-6c144a7aa52e","Type":"ContainerStarted","Data":"f788cf77e7195aadc7058931b559278f77f304732c3b5664f84e45cda4c8b795"} Sep 19 05:48:17.318864 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:17.318827 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" event={"ID":"49a6b7a0-a424-4d6d-9875-4f8ddadc949e","Type":"ContainerStarted","Data":"28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8"} Sep 19 05:48:17.320602 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:17.320568 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-bn499" event={"ID":"369027fc-839c-4973-8009-6c144a7aa52e","Type":"ContainerStarted","Data":"5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154"} Sep 19 05:48:17.322283 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:17.322261 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-l25xc" event={"ID":"ba0ab1a6-6439-4a99-976f-cd2aae40b137","Type":"ContainerStarted","Data":"163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d"} Sep 19 05:48:17.322438 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:17.322421 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-l25xc" Sep 19 05:48:17.334170 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:17.334121 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" podStartSLOduration=1.910224159 podStartE2EDuration="4.334104286s" totalImagesPullingTime="2.423880127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:48:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:48:17.333495609 +0000 UTC m=+425.326723141" watchObservedRunningTime="2026-09-19 05:48:17.334104286 +0000 UTC m=+425.327331819" Sep 19 05:48:17.353693 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:17.353641 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-l25xc" podStartSLOduration=2.251638657 podStartE2EDuration="5.353620944s" totalImagesPullingTime="3.101982287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:48:17.352311427 +0000 UTC m=+425.345538970" watchObservedRunningTime="2026-09-19 05:48:17.353620944 +0000 UTC m=+425.346848477" Sep 19 05:48:17.367722 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:17.367659 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-bn499" podStartSLOduration=2.151402333 podStartE2EDuration="4.367642136s" totalImagesPullingTime="2.216239803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:48:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:48:17.367040067 +0000 UTC m=+425.360267598" watchObservedRunningTime="2026-09-19 05:48:17.367642136 +0000 UTC m=+425.360869666" Sep 19 05:48:17.392632 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:17.392598 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lmhw7"] Sep 19 05:48:19.335069 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:19.335035 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" podUID="49a6b7a0-a424-4d6d-9875-4f8ddadc949e" containerName="authorino" containerID="cri-o://28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8" gracePeriod=30 Sep 19 05:48:19.584319 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:19.584294 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" Sep 19 05:48:19.724331 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:19.724234 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnnfk\" (UniqueName: \"kubernetes.io/projected/49a6b7a0-a424-4d6d-9875-4f8ddadc949e-kube-api-access-xnnfk\") pod \"49a6b7a0-a424-4d6d-9875-4f8ddadc949e\" (UID: \"49a6b7a0-a424-4d6d-9875-4f8ddadc949e\") " Sep 19 05:48:19.726416 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:19.726370 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49a6b7a0-a424-4d6d-9875-4f8ddadc949e-kube-api-access-xnnfk" (OuterVolumeSpecName: "kube-api-access-xnnfk") pod "49a6b7a0-a424-4d6d-9875-4f8ddadc949e" (UID: "49a6b7a0-a424-4d6d-9875-4f8ddadc949e"). InnerVolumeSpecName "kube-api-access-xnnfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:48:19.825244 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:19.825202 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnnfk\" (UniqueName: \"kubernetes.io/projected/49a6b7a0-a424-4d6d-9875-4f8ddadc949e-kube-api-access-xnnfk\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:48:20.341170 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:20.341135 2584 generic.go:356] "Generic (PLEG): container finished" podID="49a6b7a0-a424-4d6d-9875-4f8ddadc949e" containerID="28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8" exitCode=0 Sep 19 05:48:20.341596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:20.341199 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" Sep 19 05:48:20.341596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:20.341238 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" event={"ID":"49a6b7a0-a424-4d6d-9875-4f8ddadc949e","Type":"ContainerDied","Data":"28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8"} Sep 19 05:48:20.341596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:20.341270 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lmhw7" event={"ID":"49a6b7a0-a424-4d6d-9875-4f8ddadc949e","Type":"ContainerDied","Data":"3c8a67d6d2035e60fe3b934bacad9ab5d6cd7d3f582b92bf1d2ccc25140ea53b"} Sep 19 05:48:20.341596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:20.341287 2584 scope.go:117] "RemoveContainer" containerID="28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8" Sep 19 05:48:20.349623 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:20.349607 2584 scope.go:117] "RemoveContainer" containerID="28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8" Sep 19 05:48:20.349910 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:48:20.349892 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8\": container with ID starting with 28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8 not found: ID does not exist" containerID="28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8" Sep 19 05:48:20.349952 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:20.349917 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8"} err="failed to get container status \"28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8\": rpc error: code = NotFound desc = could not find container \"28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8\": container with ID starting with 28c9cdc49c47d1741362bd68867ab7afc3fc1558f268df0488031a292edf55d8 not found: ID does not exist" Sep 19 05:48:20.362038 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:20.362003 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lmhw7"] Sep 19 05:48:20.365605 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:20.365580 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lmhw7"] Sep 19 05:48:22.266255 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:22.266208 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49a6b7a0-a424-4d6d-9875-4f8ddadc949e" path="/var/lib/kubelet/pods/49a6b7a0-a424-4d6d-9875-4f8ddadc949e/volumes" Sep 19 05:48:26.432135 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:26.432048 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-l25xc"] Sep 19 05:48:26.432613 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:26.432253 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-l25xc" podUID="ba0ab1a6-6439-4a99-976f-cd2aae40b137" containerName="limitador" containerID="cri-o://163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d" gracePeriod=30 Sep 19 05:48:26.433012 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:26.432990 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-l25xc" Sep 19 05:48:26.974729 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:26.974701 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-l25xc" Sep 19 05:48:27.096607 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.096571 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6m9r\" (UniqueName: \"kubernetes.io/projected/ba0ab1a6-6439-4a99-976f-cd2aae40b137-kube-api-access-b6m9r\") pod \"ba0ab1a6-6439-4a99-976f-cd2aae40b137\" (UID: \"ba0ab1a6-6439-4a99-976f-cd2aae40b137\") " Sep 19 05:48:27.096776 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.096632 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ba0ab1a6-6439-4a99-976f-cd2aae40b137-config-file\") pod \"ba0ab1a6-6439-4a99-976f-cd2aae40b137\" (UID: \"ba0ab1a6-6439-4a99-976f-cd2aae40b137\") " Sep 19 05:48:27.097061 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.097034 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba0ab1a6-6439-4a99-976f-cd2aae40b137-config-file" (OuterVolumeSpecName: "config-file") pod "ba0ab1a6-6439-4a99-976f-cd2aae40b137" (UID: "ba0ab1a6-6439-4a99-976f-cd2aae40b137"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:48:27.098787 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.098760 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba0ab1a6-6439-4a99-976f-cd2aae40b137-kube-api-access-b6m9r" (OuterVolumeSpecName: "kube-api-access-b6m9r") pod "ba0ab1a6-6439-4a99-976f-cd2aae40b137" (UID: "ba0ab1a6-6439-4a99-976f-cd2aae40b137"). InnerVolumeSpecName "kube-api-access-b6m9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:48:27.197813 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.197762 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6m9r\" (UniqueName: \"kubernetes.io/projected/ba0ab1a6-6439-4a99-976f-cd2aae40b137-kube-api-access-b6m9r\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:48:27.197813 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.197807 2584 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ba0ab1a6-6439-4a99-976f-cd2aae40b137-config-file\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:48:27.376831 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.376749 2584 generic.go:356] "Generic (PLEG): container finished" podID="ba0ab1a6-6439-4a99-976f-cd2aae40b137" containerID="163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d" exitCode=0 Sep 19 05:48:27.376831 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.376796 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-l25xc" Sep 19 05:48:27.377107 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.376835 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-l25xc" event={"ID":"ba0ab1a6-6439-4a99-976f-cd2aae40b137","Type":"ContainerDied","Data":"163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d"} Sep 19 05:48:27.377107 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.376884 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-l25xc" event={"ID":"ba0ab1a6-6439-4a99-976f-cd2aae40b137","Type":"ContainerDied","Data":"f31b781f110fd4b523db29c61304ebb7b996286bd5a3f715c5279382bce45c9c"} Sep 19 05:48:27.377107 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.376905 2584 scope.go:117] "RemoveContainer" containerID="163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d" Sep 19 05:48:27.385460 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.385441 2584 scope.go:117] "RemoveContainer" containerID="163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d" Sep 19 05:48:27.385718 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:48:27.385701 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d\": container with ID starting with 163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d not found: ID does not exist" containerID="163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d" Sep 19 05:48:27.385781 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.385725 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d"} err="failed to get container status \"163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d\": rpc error: code = NotFound desc = could not find container \"163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d\": container with ID starting with 163ab7ebe56903486fcc59e604198d4149a0c7d678f0556f06b34220bffea69d not found: ID does not exist" Sep 19 05:48:27.397895 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.397865 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-l25xc"] Sep 19 05:48:27.400280 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:27.400256 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-l25xc"] Sep 19 05:48:28.266967 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:28.266935 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba0ab1a6-6439-4a99-976f-cd2aae40b137" path="/var/lib/kubelet/pods/ba0ab1a6-6439-4a99-976f-cd2aae40b137/volumes" Sep 19 05:48:29.036145 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.036108 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:29.134318 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.134283 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:29.481395 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.481359 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q"] Sep 19 05:48:29.481787 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.481768 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba0ab1a6-6439-4a99-976f-cd2aae40b137" containerName="limitador" Sep 19 05:48:29.481787 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.481780 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba0ab1a6-6439-4a99-976f-cd2aae40b137" containerName="limitador" Sep 19 05:48:29.481858 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.481800 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49a6b7a0-a424-4d6d-9875-4f8ddadc949e" containerName="authorino" Sep 19 05:48:29.481858 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.481806 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="49a6b7a0-a424-4d6d-9875-4f8ddadc949e" containerName="authorino" Sep 19 05:48:29.481917 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.481897 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba0ab1a6-6439-4a99-976f-cd2aae40b137" containerName="limitador" Sep 19 05:48:29.481917 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.481909 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="49a6b7a0-a424-4d6d-9875-4f8ddadc949e" containerName="authorino" Sep 19 05:48:29.486710 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.486685 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" Sep 19 05:48:29.489180 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.489158 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 19 05:48:29.489365 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.489277 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 19 05:48:29.490173 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.490154 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 19 05:48:29.490283 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.490191 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-n5q69\"" Sep 19 05:48:29.492226 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.492204 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q"] Sep 19 05:48:29.540939 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.540905 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:29.619976 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.619929 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ebfa06ac-808c-4082-a57d-f287453b462f-data\") pod \"postgres-69fb4594cf-rzc2q\" (UID: \"ebfa06ac-808c-4082-a57d-f287453b462f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" Sep 19 05:48:29.620145 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.620088 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcdkr\" (UniqueName: \"kubernetes.io/projected/ebfa06ac-808c-4082-a57d-f287453b462f-kube-api-access-pcdkr\") pod \"postgres-69fb4594cf-rzc2q\" (UID: \"ebfa06ac-808c-4082-a57d-f287453b462f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" Sep 19 05:48:29.721352 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.721310 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcdkr\" (UniqueName: \"kubernetes.io/projected/ebfa06ac-808c-4082-a57d-f287453b462f-kube-api-access-pcdkr\") pod \"postgres-69fb4594cf-rzc2q\" (UID: \"ebfa06ac-808c-4082-a57d-f287453b462f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" Sep 19 05:48:29.721352 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.721355 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ebfa06ac-808c-4082-a57d-f287453b462f-data\") pod \"postgres-69fb4594cf-rzc2q\" (UID: \"ebfa06ac-808c-4082-a57d-f287453b462f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" Sep 19 05:48:29.721709 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.721692 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ebfa06ac-808c-4082-a57d-f287453b462f-data\") pod \"postgres-69fb4594cf-rzc2q\" (UID: \"ebfa06ac-808c-4082-a57d-f287453b462f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" Sep 19 05:48:29.744687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.744619 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcdkr\" (UniqueName: \"kubernetes.io/projected/ebfa06ac-808c-4082-a57d-f287453b462f-kube-api-access-pcdkr\") pod \"postgres-69fb4594cf-rzc2q\" (UID: \"ebfa06ac-808c-4082-a57d-f287453b462f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" Sep 19 05:48:29.798181 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:29.798144 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" Sep 19 05:48:30.077289 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:30.077263 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q"] Sep 19 05:48:30.078952 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:48:30.078925 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfa06ac_808c_4082_a57d_f287453b462f.slice/crio-479f5c3271e262a7b0ef74ab36d6241577c159596320f1849390094316080f09 WatchSource:0}: Error finding container 479f5c3271e262a7b0ef74ab36d6241577c159596320f1849390094316080f09: Status 404 returned error can't find the container with id 479f5c3271e262a7b0ef74ab36d6241577c159596320f1849390094316080f09 Sep 19 05:48:30.396428 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:30.396373 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" event={"ID":"ebfa06ac-808c-4082-a57d-f287453b462f","Type":"ContainerStarted","Data":"479f5c3271e262a7b0ef74ab36d6241577c159596320f1849390094316080f09"} Sep 19 05:48:35.424773 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:35.424733 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" event={"ID":"ebfa06ac-808c-4082-a57d-f287453b462f","Type":"ContainerStarted","Data":"f9020257e482af7fe8ef788aa2c3ad4d2f66d46d474a6b789889ea3da599ccd0"} Sep 19 05:48:35.425162 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:35.424873 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" Sep 19 05:48:35.441264 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:35.441216 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" podStartSLOduration=1.5847130470000002 podStartE2EDuration="6.441201946s" totalImagesPullingTime="4.856488899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:48:35.439738062 +0000 UTC m=+443.432965593" watchObservedRunningTime="2026-09-19 05:48:35.441201946 +0000 UTC m=+443.434429473" Sep 19 05:48:41.459086 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:41.459054 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzc2q" Sep 19 05:48:41.789192 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:41.789104 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:42.033369 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.033322 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-psn26"] Sep 19 05:48:42.039332 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.039266 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-psn26" Sep 19 05:48:42.045092 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.045067 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-psn26"] Sep 19 05:48:42.135693 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.135644 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ssh5\" (UniqueName: \"kubernetes.io/projected/ee0132af-b127-4bb4-a4c1-caf884eb3544-kube-api-access-9ssh5\") pod \"authorino-557f876f69-psn26\" (UID: \"ee0132af-b127-4bb4-a4c1-caf884eb3544\") " pod="kuadrant-system/authorino-557f876f69-psn26" Sep 19 05:48:42.236964 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.236925 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ssh5\" (UniqueName: \"kubernetes.io/projected/ee0132af-b127-4bb4-a4c1-caf884eb3544-kube-api-access-9ssh5\") pod \"authorino-557f876f69-psn26\" (UID: \"ee0132af-b127-4bb4-a4c1-caf884eb3544\") " pod="kuadrant-system/authorino-557f876f69-psn26" Sep 19 05:48:42.246329 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.246299 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ssh5\" (UniqueName: \"kubernetes.io/projected/ee0132af-b127-4bb4-a4c1-caf884eb3544-kube-api-access-9ssh5\") pod \"authorino-557f876f69-psn26\" (UID: \"ee0132af-b127-4bb4-a4c1-caf884eb3544\") " pod="kuadrant-system/authorino-557f876f69-psn26" Sep 19 05:48:42.291021 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.290940 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-psn26"] Sep 19 05:48:42.291224 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.291211 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-psn26" Sep 19 05:48:42.302271 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.302242 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:42.420636 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.420601 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c4b6f868b-qwrdp"] Sep 19 05:48:42.426267 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.426245 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" Sep 19 05:48:42.429008 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.428988 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 19 05:48:42.435049 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.435022 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c4b6f868b-qwrdp"] Sep 19 05:48:42.539154 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.539121 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/749384e6-4ccf-41b9-8e4f-bfa807a7aab5-tls-cert\") pod \"authorino-6c4b6f868b-qwrdp\" (UID: \"749384e6-4ccf-41b9-8e4f-bfa807a7aab5\") " pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" Sep 19 05:48:42.539676 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.539253 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q72s\" (UniqueName: \"kubernetes.io/projected/749384e6-4ccf-41b9-8e4f-bfa807a7aab5-kube-api-access-4q72s\") pod \"authorino-6c4b6f868b-qwrdp\" (UID: \"749384e6-4ccf-41b9-8e4f-bfa807a7aab5\") " pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" Sep 19 05:48:42.571855 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.571829 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-psn26"] Sep 19 05:48:42.573798 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:48:42.573766 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee0132af_b127_4bb4_a4c1_caf884eb3544.slice/crio-c26c16aa59a6604d510855c48d3780fbb25ca4df9932d85054eb1014c1e9e3fb WatchSource:0}: Error finding container c26c16aa59a6604d510855c48d3780fbb25ca4df9932d85054eb1014c1e9e3fb: Status 404 returned error can't find the container with id c26c16aa59a6604d510855c48d3780fbb25ca4df9932d85054eb1014c1e9e3fb Sep 19 05:48:42.640364 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.640322 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q72s\" (UniqueName: \"kubernetes.io/projected/749384e6-4ccf-41b9-8e4f-bfa807a7aab5-kube-api-access-4q72s\") pod \"authorino-6c4b6f868b-qwrdp\" (UID: \"749384e6-4ccf-41b9-8e4f-bfa807a7aab5\") " pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" Sep 19 05:48:42.640571 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.640380 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/749384e6-4ccf-41b9-8e4f-bfa807a7aab5-tls-cert\") pod \"authorino-6c4b6f868b-qwrdp\" (UID: \"749384e6-4ccf-41b9-8e4f-bfa807a7aab5\") " pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" Sep 19 05:48:42.642825 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.642802 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/749384e6-4ccf-41b9-8e4f-bfa807a7aab5-tls-cert\") pod \"authorino-6c4b6f868b-qwrdp\" (UID: \"749384e6-4ccf-41b9-8e4f-bfa807a7aab5\") " pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" Sep 19 05:48:42.651123 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.651099 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q72s\" (UniqueName: \"kubernetes.io/projected/749384e6-4ccf-41b9-8e4f-bfa807a7aab5-kube-api-access-4q72s\") pod \"authorino-6c4b6f868b-qwrdp\" (UID: \"749384e6-4ccf-41b9-8e4f-bfa807a7aab5\") " pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" Sep 19 05:48:42.737232 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:42.737186 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" Sep 19 05:48:43.011236 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.011210 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c4b6f868b-qwrdp"] Sep 19 05:48:43.013228 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:48:43.013209 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod749384e6_4ccf_41b9_8e4f_bfa807a7aab5.slice/crio-9349e484405755b24916cbf74d19771445385e2eb7dfc7d54cb6ce93e261dfaf WatchSource:0}: Error finding container 9349e484405755b24916cbf74d19771445385e2eb7dfc7d54cb6ce93e261dfaf: Status 404 returned error can't find the container with id 9349e484405755b24916cbf74d19771445385e2eb7dfc7d54cb6ce93e261dfaf Sep 19 05:48:43.465426 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.465355 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-psn26" event={"ID":"ee0132af-b127-4bb4-a4c1-caf884eb3544","Type":"ContainerStarted","Data":"8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06"} Sep 19 05:48:43.465603 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.465438 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-psn26" event={"ID":"ee0132af-b127-4bb4-a4c1-caf884eb3544","Type":"ContainerStarted","Data":"c26c16aa59a6604d510855c48d3780fbb25ca4df9932d85054eb1014c1e9e3fb"} Sep 19 05:48:43.465603 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.465476 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-psn26" podUID="ee0132af-b127-4bb4-a4c1-caf884eb3544" containerName="authorino" containerID="cri-o://8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06" gracePeriod=30 Sep 19 05:48:43.467405 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.467374 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" event={"ID":"749384e6-4ccf-41b9-8e4f-bfa807a7aab5","Type":"ContainerStarted","Data":"8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3"} Sep 19 05:48:43.467539 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.467431 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" event={"ID":"749384e6-4ccf-41b9-8e4f-bfa807a7aab5","Type":"ContainerStarted","Data":"9349e484405755b24916cbf74d19771445385e2eb7dfc7d54cb6ce93e261dfaf"} Sep 19 05:48:43.484748 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.484700 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-psn26" podStartSLOduration=0.958741433 podStartE2EDuration="1.484684346s" totalImagesPullingTime="525.942913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:48:43.483208564 +0000 UTC m=+451.476436094" watchObservedRunningTime="2026-09-19 05:48:43.484684346 +0000 UTC m=+451.477911877" Sep 19 05:48:43.506734 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.506680 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" podStartSLOduration=1.219679042 podStartE2EDuration="1.506662543s" totalImagesPullingTime="286.983501ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:48:43.505068364 +0000 UTC m=+451.498295895" watchObservedRunningTime="2026-09-19 05:48:43.506662543 +0000 UTC m=+451.499890073" Sep 19 05:48:43.545920 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.545881 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-bn499"] Sep 19 05:48:43.546331 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.546076 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-bn499" podUID="369027fc-839c-4973-8009-6c144a7aa52e" containerName="authorino" containerID="cri-o://5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154" gracePeriod=30 Sep 19 05:48:43.793190 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.793169 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-psn26" Sep 19 05:48:43.799820 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.799797 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-bn499" Sep 19 05:48:43.851727 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.851692 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ssh5\" (UniqueName: \"kubernetes.io/projected/ee0132af-b127-4bb4-a4c1-caf884eb3544-kube-api-access-9ssh5\") pod \"ee0132af-b127-4bb4-a4c1-caf884eb3544\" (UID: \"ee0132af-b127-4bb4-a4c1-caf884eb3544\") " Sep 19 05:48:43.851912 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.851864 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wv4z\" (UniqueName: \"kubernetes.io/projected/369027fc-839c-4973-8009-6c144a7aa52e-kube-api-access-2wv4z\") pod \"369027fc-839c-4973-8009-6c144a7aa52e\" (UID: \"369027fc-839c-4973-8009-6c144a7aa52e\") " Sep 19 05:48:43.853895 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.853862 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/369027fc-839c-4973-8009-6c144a7aa52e-kube-api-access-2wv4z" (OuterVolumeSpecName: "kube-api-access-2wv4z") pod "369027fc-839c-4973-8009-6c144a7aa52e" (UID: "369027fc-839c-4973-8009-6c144a7aa52e"). InnerVolumeSpecName "kube-api-access-2wv4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:48:43.854059 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.853865 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee0132af-b127-4bb4-a4c1-caf884eb3544-kube-api-access-9ssh5" (OuterVolumeSpecName: "kube-api-access-9ssh5") pod "ee0132af-b127-4bb4-a4c1-caf884eb3544" (UID: "ee0132af-b127-4bb4-a4c1-caf884eb3544"). InnerVolumeSpecName "kube-api-access-9ssh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:48:43.953252 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.953219 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wv4z\" (UniqueName: \"kubernetes.io/projected/369027fc-839c-4973-8009-6c144a7aa52e-kube-api-access-2wv4z\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:48:43.953252 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:43.953246 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ssh5\" (UniqueName: \"kubernetes.io/projected/ee0132af-b127-4bb4-a4c1-caf884eb3544-kube-api-access-9ssh5\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:48:44.475855 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.475822 2584 generic.go:356] "Generic (PLEG): container finished" podID="369027fc-839c-4973-8009-6c144a7aa52e" containerID="5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154" exitCode=0 Sep 19 05:48:44.476040 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.475867 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-bn499" Sep 19 05:48:44.476040 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.475903 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-bn499" event={"ID":"369027fc-839c-4973-8009-6c144a7aa52e","Type":"ContainerDied","Data":"5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154"} Sep 19 05:48:44.476040 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.475942 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-bn499" event={"ID":"369027fc-839c-4973-8009-6c144a7aa52e","Type":"ContainerDied","Data":"f788cf77e7195aadc7058931b559278f77f304732c3b5664f84e45cda4c8b795"} Sep 19 05:48:44.476040 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.475959 2584 scope.go:117] "RemoveContainer" containerID="5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154" Sep 19 05:48:44.477314 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.477294 2584 generic.go:356] "Generic (PLEG): container finished" podID="ee0132af-b127-4bb4-a4c1-caf884eb3544" containerID="8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06" exitCode=0 Sep 19 05:48:44.477389 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.477327 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-psn26" event={"ID":"ee0132af-b127-4bb4-a4c1-caf884eb3544","Type":"ContainerDied","Data":"8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06"} Sep 19 05:48:44.477389 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.477348 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-psn26" Sep 19 05:48:44.477389 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.477361 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-psn26" event={"ID":"ee0132af-b127-4bb4-a4c1-caf884eb3544","Type":"ContainerDied","Data":"c26c16aa59a6604d510855c48d3780fbb25ca4df9932d85054eb1014c1e9e3fb"} Sep 19 05:48:44.484333 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.484318 2584 scope.go:117] "RemoveContainer" containerID="5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154" Sep 19 05:48:44.484641 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:48:44.484622 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154\": container with ID starting with 5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154 not found: ID does not exist" containerID="5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154" Sep 19 05:48:44.484728 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.484649 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154"} err="failed to get container status \"5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154\": rpc error: code = NotFound desc = could not find container \"5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154\": container with ID starting with 5e075baf10e1ac78c00a236a733083d6e0ede9f9ccb6bd227275a9052d7a4154 not found: ID does not exist" Sep 19 05:48:44.484728 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.484665 2584 scope.go:117] "RemoveContainer" containerID="8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06" Sep 19 05:48:44.492573 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.492469 2584 scope.go:117] "RemoveContainer" containerID="8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06" Sep 19 05:48:44.492845 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:48:44.492818 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06\": container with ID starting with 8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06 not found: ID does not exist" containerID="8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06" Sep 19 05:48:44.492919 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.492847 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06"} err="failed to get container status \"8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06\": rpc error: code = NotFound desc = could not find container \"8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06\": container with ID starting with 8f1257c1cf252d191e6ff96c9c0df9400391518d881da2f820546ba6de277d06 not found: ID does not exist" Sep 19 05:48:44.494308 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.494283 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-bn499"] Sep 19 05:48:44.503310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.503242 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-bn499"] Sep 19 05:48:44.513934 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.513905 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-psn26"] Sep 19 05:48:44.518542 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:44.518516 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-psn26"] Sep 19 05:48:46.267165 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:46.267133 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="369027fc-839c-4973-8009-6c144a7aa52e" path="/var/lib/kubelet/pods/369027fc-839c-4973-8009-6c144a7aa52e/volumes" Sep 19 05:48:46.267541 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:46.267498 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee0132af-b127-4bb4-a4c1-caf884eb3544" path="/var/lib/kubelet/pods/ee0132af-b127-4bb4-a4c1-caf884eb3544/volumes" Sep 19 05:48:48.649423 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.649377 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:48.681954 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.681920 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b7fb6b9f7-9k4p6"] Sep 19 05:48:48.682328 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.682313 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee0132af-b127-4bb4-a4c1-caf884eb3544" containerName="authorino" Sep 19 05:48:48.682376 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.682333 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee0132af-b127-4bb4-a4c1-caf884eb3544" containerName="authorino" Sep 19 05:48:48.682376 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.682353 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="369027fc-839c-4973-8009-6c144a7aa52e" containerName="authorino" Sep 19 05:48:48.682376 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.682363 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="369027fc-839c-4973-8009-6c144a7aa52e" containerName="authorino" Sep 19 05:48:48.682500 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.682488 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee0132af-b127-4bb4-a4c1-caf884eb3544" containerName="authorino" Sep 19 05:48:48.682540 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.682508 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="369027fc-839c-4973-8009-6c144a7aa52e" containerName="authorino" Sep 19 05:48:48.744374 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.744347 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:48.811442 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.811414 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:48.814198 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.814180 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b7fb6b9f7-9k4p6"] Sep 19 05:48:48.814303 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.814288 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:48.816783 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.816762 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 19 05:48:48.816920 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.816832 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 19 05:48:48.816920 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.816853 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-znct9\"" Sep 19 05:48:48.816920 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.816881 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 19 05:48:48.857091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.857056 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-687fc9b78d-xgndz"] Sep 19 05:48:48.885560 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.885530 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-687fc9b78d-xgndz"] Sep 19 05:48:48.885706 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.885653 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:48.894987 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.894962 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-metrics-certs\") pod \"maas-controller-b7fb6b9f7-9k4p6\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:48.895108 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.895025 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mddrp\" (UniqueName: \"kubernetes.io/projected/54732ef5-36bf-44f8-932d-6f3aea6f359f-kube-api-access-mddrp\") pod \"maas-controller-b7fb6b9f7-9k4p6\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:48.895108 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.895078 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-webhook-cert\") pod \"maas-controller-b7fb6b9f7-9k4p6\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:48.990132 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.990054 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:48.996274 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.996243 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mddrp\" (UniqueName: \"kubernetes.io/projected/54732ef5-36bf-44f8-932d-6f3aea6f359f-kube-api-access-mddrp\") pod \"maas-controller-b7fb6b9f7-9k4p6\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:48.996676 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.996309 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-metrics-certs\") pod \"maas-controller-687fc9b78d-xgndz\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:48.996676 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.996363 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-webhook-cert\") pod \"maas-controller-b7fb6b9f7-9k4p6\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:48.996676 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.996425 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt9mv\" (UniqueName: \"kubernetes.io/projected/b23c027b-c65f-49aa-82c3-834df63718e5-kube-api-access-kt9mv\") pod \"maas-controller-687fc9b78d-xgndz\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:48.996676 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.996467 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-webhook-cert\") pod \"maas-controller-687fc9b78d-xgndz\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:48.996676 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:48:48.996496 2584 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 19 05:48:48.996676 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.996534 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-metrics-certs\") pod \"maas-controller-b7fb6b9f7-9k4p6\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:48.996676 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:48:48.996562 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-webhook-cert podName:54732ef5-36bf-44f8-932d-6f3aea6f359f nodeName:}" failed. No retries permitted until 2026-09-19 05:48:49.496540087 +0000 UTC m=+457.489767607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-webhook-cert") pod "maas-controller-b7fb6b9f7-9k4p6" (UID: "54732ef5-36bf-44f8-932d-6f3aea6f359f") : secret "maas-controller-webhook-cert" not found Sep 19 05:48:48.998966 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:48.998947 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-metrics-certs\") pod \"maas-controller-b7fb6b9f7-9k4p6\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:49.005495 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.005474 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mddrp\" (UniqueName: \"kubernetes.io/projected/54732ef5-36bf-44f8-932d-6f3aea6f359f-kube-api-access-mddrp\") pod \"maas-controller-b7fb6b9f7-9k4p6\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:49.097803 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.097764 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-metrics-certs\") pod \"maas-controller-687fc9b78d-xgndz\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:49.098010 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.097829 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt9mv\" (UniqueName: \"kubernetes.io/projected/b23c027b-c65f-49aa-82c3-834df63718e5-kube-api-access-kt9mv\") pod \"maas-controller-687fc9b78d-xgndz\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:49.098010 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.097944 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-webhook-cert\") pod \"maas-controller-687fc9b78d-xgndz\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:49.098126 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:48:49.098064 2584 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 19 05:48:49.098179 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:48:49.098133 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-webhook-cert podName:b23c027b-c65f-49aa-82c3-834df63718e5 nodeName:}" failed. No retries permitted until 2026-09-19 05:48:49.598118061 +0000 UTC m=+457.591345573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-webhook-cert") pod "maas-controller-687fc9b78d-xgndz" (UID: "b23c027b-c65f-49aa-82c3-834df63718e5") : secret "maas-controller-webhook-cert" not found Sep 19 05:48:49.100369 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.100347 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-metrics-certs\") pod \"maas-controller-687fc9b78d-xgndz\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:49.109275 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.109252 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt9mv\" (UniqueName: \"kubernetes.io/projected/b23c027b-c65f-49aa-82c3-834df63718e5-kube-api-access-kt9mv\") pod \"maas-controller-687fc9b78d-xgndz\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:49.358033 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.358000 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:49.489317 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.489286 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:49.500740 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.500716 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-webhook-cert\") pod \"maas-controller-b7fb6b9f7-9k4p6\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:49.503385 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.503359 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-webhook-cert\") pod \"maas-controller-b7fb6b9f7-9k4p6\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:49.556441 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.556388 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:49.601450 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.601390 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-webhook-cert\") pod \"maas-controller-687fc9b78d-xgndz\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:49.603729 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.603703 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-webhook-cert\") pod \"maas-controller-687fc9b78d-xgndz\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:49.724848 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.724758 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:49.796071 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:49.795828 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:50.013173 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:50.013100 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b7fb6b9f7-9k4p6"] Sep 19 05:48:50.015858 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:48:50.015817 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54732ef5_36bf_44f8_932d_6f3aea6f359f.slice/crio-f3b2699e1d18454c72a33ff45c41a0de9ff5271e60eb67c2919f29123ad514a5 WatchSource:0}: Error finding container f3b2699e1d18454c72a33ff45c41a0de9ff5271e60eb67c2919f29123ad514a5: Status 404 returned error can't find the container with id f3b2699e1d18454c72a33ff45c41a0de9ff5271e60eb67c2919f29123ad514a5 Sep 19 05:48:50.092165 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:50.092143 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-687fc9b78d-xgndz"] Sep 19 05:48:50.094717 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:48:50.094691 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23c027b_c65f_49aa_82c3_834df63718e5.slice/crio-376577728b80c69b88a706dc53b85c88c5c579313d4f2b3f36f39ad82ae2f88b WatchSource:0}: Error finding container 376577728b80c69b88a706dc53b85c88c5c579313d4f2b3f36f39ad82ae2f88b: Status 404 returned error can't find the container with id 376577728b80c69b88a706dc53b85c88c5c579313d4f2b3f36f39ad82ae2f88b Sep 19 05:48:50.510353 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:50.510318 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687fc9b78d-xgndz" event={"ID":"b23c027b-c65f-49aa-82c3-834df63718e5","Type":"ContainerStarted","Data":"376577728b80c69b88a706dc53b85c88c5c579313d4f2b3f36f39ad82ae2f88b"} Sep 19 05:48:50.511524 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:50.511498 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" event={"ID":"54732ef5-36bf-44f8-932d-6f3aea6f359f","Type":"ContainerStarted","Data":"f3b2699e1d18454c72a33ff45c41a0de9ff5271e60eb67c2919f29123ad514a5"} Sep 19 05:48:52.865827 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:52.865793 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:52.979254 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:52.979151 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:48:53.532478 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:53.532438 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687fc9b78d-xgndz" event={"ID":"b23c027b-c65f-49aa-82c3-834df63718e5","Type":"ContainerStarted","Data":"6529a2afa91f82238a41c9d8ea7a8620c803a2837f87a0b547b9a8fcd449ef9e"} Sep 19 05:48:53.532695 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:53.532549 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:48:53.534268 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:53.534238 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" event={"ID":"54732ef5-36bf-44f8-932d-6f3aea6f359f","Type":"ContainerStarted","Data":"2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef"} Sep 19 05:48:53.534418 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:53.534357 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:48:53.561701 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:53.561645 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-687fc9b78d-xgndz" podStartSLOduration=2.991792018 podStartE2EDuration="5.561629001s" totalImagesPullingTime="2.569836983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:48:53.559879827 +0000 UTC m=+461.553107372" watchObservedRunningTime="2026-09-19 05:48:53.561629001 +0000 UTC m=+461.554856531" Sep 19 05:48:53.589624 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:48:53.589576 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" podStartSLOduration=2.951353642 podStartE2EDuration="5.589562676s" totalImagesPullingTime="2.638209034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:48:53.587896192 +0000 UTC m=+461.581123724" watchObservedRunningTime="2026-09-19 05:48:53.589562676 +0000 UTC m=+461.582790207" Sep 19 05:49:04.541475 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:04.541442 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:49:04.541948 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:04.541619 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 05:49:04.608580 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:04.608548 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b7fb6b9f7-9k4p6"] Sep 19 05:49:04.608786 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:04.608760 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" podUID="54732ef5-36bf-44f8-932d-6f3aea6f359f" containerName="manager" containerID="cri-o://2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef" gracePeriod=30 Sep 19 05:49:04.851226 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:04.851198 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:49:04.952171 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:04.952135 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mddrp\" (UniqueName: \"kubernetes.io/projected/54732ef5-36bf-44f8-932d-6f3aea6f359f-kube-api-access-mddrp\") pod \"54732ef5-36bf-44f8-932d-6f3aea6f359f\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " Sep 19 05:49:04.952381 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:04.952199 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-metrics-certs\") pod \"54732ef5-36bf-44f8-932d-6f3aea6f359f\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " Sep 19 05:49:04.952381 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:04.952252 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-webhook-cert\") pod \"54732ef5-36bf-44f8-932d-6f3aea6f359f\" (UID: \"54732ef5-36bf-44f8-932d-6f3aea6f359f\") " Sep 19 05:49:04.954563 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:04.954534 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54732ef5-36bf-44f8-932d-6f3aea6f359f-kube-api-access-mddrp" (OuterVolumeSpecName: "kube-api-access-mddrp") pod "54732ef5-36bf-44f8-932d-6f3aea6f359f" (UID: "54732ef5-36bf-44f8-932d-6f3aea6f359f"). InnerVolumeSpecName "kube-api-access-mddrp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:49:04.954563 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:04.954547 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "54732ef5-36bf-44f8-932d-6f3aea6f359f" (UID: "54732ef5-36bf-44f8-932d-6f3aea6f359f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:49:04.954719 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:04.954534 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "54732ef5-36bf-44f8-932d-6f3aea6f359f" (UID: "54732ef5-36bf-44f8-932d-6f3aea6f359f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:49:05.053070 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:05.053037 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mddrp\" (UniqueName: \"kubernetes.io/projected/54732ef5-36bf-44f8-932d-6f3aea6f359f-kube-api-access-mddrp\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:49:05.053070 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:05.053068 2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-metrics-certs\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:49:05.053070 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:05.053078 2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54732ef5-36bf-44f8-932d-6f3aea6f359f-webhook-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:49:05.599907 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:05.599869 2584 generic.go:356] "Generic (PLEG): container finished" podID="54732ef5-36bf-44f8-932d-6f3aea6f359f" containerID="2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef" exitCode=0 Sep 19 05:49:05.600343 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:05.599917 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" Sep 19 05:49:05.600343 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:05.599963 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" event={"ID":"54732ef5-36bf-44f8-932d-6f3aea6f359f","Type":"ContainerDied","Data":"2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef"} Sep 19 05:49:05.600343 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:05.600018 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b7fb6b9f7-9k4p6" event={"ID":"54732ef5-36bf-44f8-932d-6f3aea6f359f","Type":"ContainerDied","Data":"f3b2699e1d18454c72a33ff45c41a0de9ff5271e60eb67c2919f29123ad514a5"} Sep 19 05:49:05.600343 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:05.600040 2584 scope.go:117] "RemoveContainer" containerID="2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef" Sep 19 05:49:05.609280 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:05.609262 2584 scope.go:117] "RemoveContainer" containerID="2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef" Sep 19 05:49:05.609629 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:49:05.609604 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef\": container with ID starting with 2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef not found: ID does not exist" containerID="2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef" Sep 19 05:49:05.609720 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:05.609637 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef"} err="failed to get container status \"2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef\": rpc error: code = NotFound desc = could not find container \"2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef\": container with ID starting with 2aa8d79d831d4baa9dee1a9651baa7aacabf0fc92bdb1d121ca011df16c501ef not found: ID does not exist" Sep 19 05:49:05.635206 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:05.635176 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b7fb6b9f7-9k4p6"] Sep 19 05:49:05.643737 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:05.643711 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b7fb6b9f7-9k4p6"] Sep 19 05:49:06.267174 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:06.267142 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54732ef5-36bf-44f8-932d-6f3aea6f359f" path="/var/lib/kubelet/pods/54732ef5-36bf-44f8-932d-6f3aea6f359f/volumes" Sep 19 05:49:24.498670 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:24.498635 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:49:24.598803 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:24.598761 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:49:26.061848 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.061811 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:49:26.161852 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.161812 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:49:26.234799 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.234767 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:49:26.394829 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.394802 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:49:26.571665 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.571636 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:49:26.667318 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.667229 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:49:26.668194 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.668174 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h"] Sep 19 05:49:26.668583 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.668558 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54732ef5-36bf-44f8-932d-6f3aea6f359f" containerName="manager" Sep 19 05:49:26.668583 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.668585 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="54732ef5-36bf-44f8-932d-6f3aea6f359f" containerName="manager" Sep 19 05:49:26.668674 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.668662 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="54732ef5-36bf-44f8-932d-6f3aea6f359f" containerName="manager" Sep 19 05:49:26.671584 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.671570 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" Sep 19 05:49:26.673620 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.673606 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 19 05:49:26.680068 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.680047 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h"] Sep 19 05:49:26.704212 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.704182 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-dmnts"] Sep 19 05:49:26.709898 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.709867 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" Sep 19 05:49:26.712465 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.712441 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-gsb57\"" Sep 19 05:49:26.718678 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.718653 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-dmnts"] Sep 19 05:49:26.756470 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.756431 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3c2b6d2-b60c-4dde-906b-7301fea6c7f5-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-dmnts\" (UID: \"e3c2b6d2-b60c-4dde-906b-7301fea6c7f5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" Sep 19 05:49:26.756655 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.756532 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4703ef-5647-4533-a88f-6443cace6aec-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-spp4h\" (UID: \"2c4703ef-5647-4533-a88f-6443cace6aec\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" Sep 19 05:49:26.756655 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.756613 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9zf7\" (UniqueName: \"kubernetes.io/projected/e3c2b6d2-b60c-4dde-906b-7301fea6c7f5-kube-api-access-p9zf7\") pod \"payload-processing-5b54b7fbb7-dmnts\" (UID: \"e3c2b6d2-b60c-4dde-906b-7301fea6c7f5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" Sep 19 05:49:26.756655 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.756640 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wsr5\" (UniqueName: \"kubernetes.io/projected/2c4703ef-5647-4533-a88f-6443cace6aec-kube-api-access-6wsr5\") pod \"payload-pre-processing-59fbc47bf8-spp4h\" (UID: \"2c4703ef-5647-4533-a88f-6443cace6aec\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" Sep 19 05:49:26.858020 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.857975 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4703ef-5647-4533-a88f-6443cace6aec-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-spp4h\" (UID: \"2c4703ef-5647-4533-a88f-6443cace6aec\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" Sep 19 05:49:26.858240 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.858146 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9zf7\" (UniqueName: \"kubernetes.io/projected/e3c2b6d2-b60c-4dde-906b-7301fea6c7f5-kube-api-access-p9zf7\") pod \"payload-processing-5b54b7fbb7-dmnts\" (UID: \"e3c2b6d2-b60c-4dde-906b-7301fea6c7f5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" Sep 19 05:49:26.858240 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.858197 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wsr5\" (UniqueName: \"kubernetes.io/projected/2c4703ef-5647-4533-a88f-6443cace6aec-kube-api-access-6wsr5\") pod \"payload-pre-processing-59fbc47bf8-spp4h\" (UID: \"2c4703ef-5647-4533-a88f-6443cace6aec\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" Sep 19 05:49:26.858240 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.858239 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3c2b6d2-b60c-4dde-906b-7301fea6c7f5-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-dmnts\" (UID: \"e3c2b6d2-b60c-4dde-906b-7301fea6c7f5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" Sep 19 05:49:26.858958 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.858936 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4703ef-5647-4533-a88f-6443cace6aec-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-spp4h\" (UID: \"2c4703ef-5647-4533-a88f-6443cace6aec\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" Sep 19 05:49:26.859103 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.859075 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3c2b6d2-b60c-4dde-906b-7301fea6c7f5-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-dmnts\" (UID: \"e3c2b6d2-b60c-4dde-906b-7301fea6c7f5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" Sep 19 05:49:26.866099 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.866074 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wsr5\" (UniqueName: \"kubernetes.io/projected/2c4703ef-5647-4533-a88f-6443cace6aec-kube-api-access-6wsr5\") pod \"payload-pre-processing-59fbc47bf8-spp4h\" (UID: \"2c4703ef-5647-4533-a88f-6443cace6aec\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" Sep 19 05:49:26.866099 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.866082 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9zf7\" (UniqueName: \"kubernetes.io/projected/e3c2b6d2-b60c-4dde-906b-7301fea6c7f5-kube-api-access-p9zf7\") pod \"payload-processing-5b54b7fbb7-dmnts\" (UID: \"e3c2b6d2-b60c-4dde-906b-7301fea6c7f5\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" Sep 19 05:49:26.983507 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:26.983418 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" Sep 19 05:49:27.023705 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:27.023671 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" Sep 19 05:49:27.267729 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:27.267702 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h"] Sep 19 05:49:27.270570 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:49:27.270542 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c4703ef_5647_4533_a88f_6443cace6aec.slice/crio-3d7300992420d6b3badf179b172a03573048d09fce00ba1cbfa932b8712187f3 WatchSource:0}: Error finding container 3d7300992420d6b3badf179b172a03573048d09fce00ba1cbfa932b8712187f3: Status 404 returned error can't find the container with id 3d7300992420d6b3badf179b172a03573048d09fce00ba1cbfa932b8712187f3 Sep 19 05:49:27.305943 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:27.305919 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-dmnts"] Sep 19 05:49:27.308110 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:49:27.308079 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3c2b6d2_b60c_4dde_906b_7301fea6c7f5.slice/crio-39c0059ea07eb53c105597fa20a60a9dbad9843dd354e782ffe64273ee9fccb6 WatchSource:0}: Error finding container 39c0059ea07eb53c105597fa20a60a9dbad9843dd354e782ffe64273ee9fccb6: Status 404 returned error can't find the container with id 39c0059ea07eb53c105597fa20a60a9dbad9843dd354e782ffe64273ee9fccb6 Sep 19 05:49:27.712734 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:27.712692 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" event={"ID":"2c4703ef-5647-4533-a88f-6443cace6aec","Type":"ContainerStarted","Data":"3d7300992420d6b3badf179b172a03573048d09fce00ba1cbfa932b8712187f3"} Sep 19 05:49:27.714012 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:27.713976 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" event={"ID":"e3c2b6d2-b60c-4dde-906b-7301fea6c7f5","Type":"ContainerStarted","Data":"39c0059ea07eb53c105597fa20a60a9dbad9843dd354e782ffe64273ee9fccb6"} Sep 19 05:49:28.722610 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:28.722166 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" event={"ID":"2c4703ef-5647-4533-a88f-6443cace6aec","Type":"ContainerStarted","Data":"bce389d91734eab0c8bfced502f2a7909c93f8104e35ad152a299591b1bb5954"} Sep 19 05:49:28.722610 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:28.722280 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" Sep 19 05:49:28.724323 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:28.724278 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" event={"ID":"e3c2b6d2-b60c-4dde-906b-7301fea6c7f5","Type":"ContainerStarted","Data":"26205e6256f0c3b23c6f65b52c734177610385fb5467259fb6788b4493e2c244"} Sep 19 05:49:28.724491 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:28.724457 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" Sep 19 05:49:28.738899 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:28.738839 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" podStartSLOduration=1.391085154 podStartE2EDuration="2.738825284s" totalImagesPullingTime="1.34774013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:49:28.738247989 +0000 UTC m=+496.731475520" watchObservedRunningTime="2026-09-19 05:49:28.738825284 +0000 UTC m=+496.732052815" Sep 19 05:49:28.758678 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:28.758621 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" podStartSLOduration=1.445109988 podStartE2EDuration="2.758600762s" totalImagesPullingTime="1.313490774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:49:28.757927729 +0000 UTC m=+496.751155255" watchObservedRunningTime="2026-09-19 05:49:28.758600762 +0000 UTC m=+496.751828293" Sep 19 05:49:38.159167 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:38.159122 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:49:38.266669 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:38.266637 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:49:39.732235 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:39.732201 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-spp4h" Sep 19 05:49:39.732738 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:39.732282 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-dmnts" Sep 19 05:49:51.250470 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.250372 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d"] Sep 19 05:49:51.255650 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.255627 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.259136 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.259105 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 19 05:49:51.259281 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.259105 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 19 05:49:51.259281 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.259106 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 19 05:49:51.259281 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.259113 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pg7v7\"" Sep 19 05:49:51.265326 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.265298 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d"] Sep 19 05:49:51.288530 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.288500 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:49:51.386139 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.386098 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qphd\" (UniqueName: \"kubernetes.io/projected/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-kube-api-access-9qphd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.386310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.386163 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.386310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.386212 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.386310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.386273 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.386310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.386302 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.386489 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.386318 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.487824 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.487785 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qphd\" (UniqueName: \"kubernetes.io/projected/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-kube-api-access-9qphd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.488037 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.487848 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.488037 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.487891 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.488037 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.487930 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.488037 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.487968 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.488037 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.487991 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.488313 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.488266 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.488313 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.488294 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.488435 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.488375 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.490264 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.490240 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.490553 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.490535 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.504209 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.504135 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qphd\" (UniqueName: \"kubernetes.io/projected/1356cd98-57e2-4fc7-acf0-4ccfa5e92e77-kube-api-access-9qphd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d\" (UID: \"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.567258 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.567221 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:49:51.851290 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:51.851249 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d"] Sep 19 05:49:51.860185 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:49:51.860160 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1356cd98_57e2_4fc7_acf0_4ccfa5e92e77.slice/crio-0e76f90a1951de750117705271da91c3d6f9fab7028c4c03fa75ad39435bd83c WatchSource:0}: Error finding container 0e76f90a1951de750117705271da91c3d6f9fab7028c4c03fa75ad39435bd83c: Status 404 returned error can't find the container with id 0e76f90a1951de750117705271da91c3d6f9fab7028c4c03fa75ad39435bd83c Sep 19 05:49:52.846043 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:52.846005 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" event={"ID":"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77","Type":"ContainerStarted","Data":"0e76f90a1951de750117705271da91c3d6f9fab7028c4c03fa75ad39435bd83c"} Sep 19 05:49:53.852143 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:53.852110 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" event={"ID":"1356cd98-57e2-4fc7-acf0-4ccfa5e92e77","Type":"ContainerStarted","Data":"92821850bee249573ab9f0ae13ae659db4ef616f6684b5070974a0932053f7f1"} Sep 19 05:49:53.869315 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:49:53.869270 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" podStartSLOduration=1.731371786 podStartE2EDuration="2.869254071s" totalImagesPullingTime="1.137882285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:49:53.867801772 +0000 UTC m=+521.861029310" watchObservedRunningTime="2026-09-19 05:49:53.869254071 +0000 UTC m=+521.862481602" Sep 19 05:50:01.567738 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:01.567641 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:50:01.567738 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:01.567701 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:50:01.580427 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:01.580376 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:50:01.902067 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:01.902036 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d" Sep 19 05:50:08.379693 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:08.379659 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:50:22.846571 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:22.846530 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:50:35.348303 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.348271 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg"] Sep 19 05:50:35.352184 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.352168 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.354534 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.354513 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 19 05:50:35.361716 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.361691 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg"] Sep 19 05:50:35.369885 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.369862 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:50:35.518634 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.518590 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff5dba78-1de8-48a7-a66c-060b364024c2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.518634 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.518636 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff5dba78-1de8-48a7-a66c-060b364024c2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.518881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.518696 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmbj8\" (UniqueName: \"kubernetes.io/projected/ff5dba78-1de8-48a7-a66c-060b364024c2-kube-api-access-jmbj8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.518881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.518755 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5dba78-1de8-48a7-a66c-060b364024c2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.518881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.518790 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff5dba78-1de8-48a7-a66c-060b364024c2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.518881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.518842 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff5dba78-1de8-48a7-a66c-060b364024c2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.620374 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.620283 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff5dba78-1de8-48a7-a66c-060b364024c2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.620374 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.620339 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff5dba78-1de8-48a7-a66c-060b364024c2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.620374 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.620379 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff5dba78-1de8-48a7-a66c-060b364024c2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.620730 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.620423 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff5dba78-1de8-48a7-a66c-060b364024c2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.620730 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.620472 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmbj8\" (UniqueName: \"kubernetes.io/projected/ff5dba78-1de8-48a7-a66c-060b364024c2-kube-api-access-jmbj8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.620730 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.620537 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5dba78-1de8-48a7-a66c-060b364024c2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.620891 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.620775 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff5dba78-1de8-48a7-a66c-060b364024c2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.620891 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.620821 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff5dba78-1de8-48a7-a66c-060b364024c2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.620962 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.620905 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff5dba78-1de8-48a7-a66c-060b364024c2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.622714 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.622685 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff5dba78-1de8-48a7-a66c-060b364024c2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.622951 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.622931 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5dba78-1de8-48a7-a66c-060b364024c2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.627868 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.627851 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmbj8\" (UniqueName: \"kubernetes.io/projected/ff5dba78-1de8-48a7-a66c-060b364024c2-kube-api-access-jmbj8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rd2mg\" (UID: \"ff5dba78-1de8-48a7-a66c-060b364024c2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.663818 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.663782 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:35.933667 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:35.933638 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg"] Sep 19 05:50:35.935146 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:50:35.935121 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff5dba78_1de8_48a7_a66c_060b364024c2.slice/crio-ff2e38a9b94373414326222b72f3b18e606672f9373618e9dee71f16e34414a1 WatchSource:0}: Error finding container ff2e38a9b94373414326222b72f3b18e606672f9373618e9dee71f16e34414a1: Status 404 returned error can't find the container with id ff2e38a9b94373414326222b72f3b18e606672f9373618e9dee71f16e34414a1 Sep 19 05:50:36.060034 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:36.059998 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" event={"ID":"ff5dba78-1de8-48a7-a66c-060b364024c2","Type":"ContainerStarted","Data":"ff2e38a9b94373414326222b72f3b18e606672f9373618e9dee71f16e34414a1"} Sep 19 05:50:37.068244 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:37.068197 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" event={"ID":"ff5dba78-1de8-48a7-a66c-060b364024c2","Type":"ContainerStarted","Data":"150751c625f3789b6eeaa410d542b3afa5bb2313c57082c9854b046091d31d7a"} Sep 19 05:50:37.088672 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:37.088606 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" podStartSLOduration=1.934800501 podStartE2EDuration="2.088583936s" totalImagesPullingTime="153.783435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:50:37.087823693 +0000 UTC m=+565.081051245" watchObservedRunningTime="2026-09-19 05:50:37.088583936 +0000 UTC m=+565.081811468" Sep 19 05:50:43.871037 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:43.871000 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:50:45.664459 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:45.664419 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:45.664881 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:45.664493 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:45.677351 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:45.677318 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:46.126439 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:46.126390 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rd2mg" Sep 19 05:50:51.292797 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:50:51.292718 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:51:02.996291 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:51:02.996251 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:51:14.172946 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:51:14.172911 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 05:51:14.175452 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:51:14.175431 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 05:52:24.820217 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:52:24.820180 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:04.321048 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:04.320963 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d6965647f-c5b7z"] Sep 19 05:53:04.324309 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:04.324291 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d6965647f-c5b7z" Sep 19 05:53:04.338945 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:04.338917 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d6965647f-c5b7z"] Sep 19 05:53:04.391513 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:04.391477 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj4m4\" (UniqueName: \"kubernetes.io/projected/6904f775-503e-4110-9747-87165a60ad3d-kube-api-access-cj4m4\") pod \"authorino-5d6965647f-c5b7z\" (UID: \"6904f775-503e-4110-9747-87165a60ad3d\") " pod="kuadrant-system/authorino-5d6965647f-c5b7z" Sep 19 05:53:04.391696 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:04.391561 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6904f775-503e-4110-9747-87165a60ad3d-tls-cert\") pod \"authorino-5d6965647f-c5b7z\" (UID: \"6904f775-503e-4110-9747-87165a60ad3d\") " pod="kuadrant-system/authorino-5d6965647f-c5b7z" Sep 19 05:53:04.492778 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:04.492738 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4m4\" (UniqueName: \"kubernetes.io/projected/6904f775-503e-4110-9747-87165a60ad3d-kube-api-access-cj4m4\") pod \"authorino-5d6965647f-c5b7z\" (UID: \"6904f775-503e-4110-9747-87165a60ad3d\") " pod="kuadrant-system/authorino-5d6965647f-c5b7z" Sep 19 05:53:04.492966 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:04.492787 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6904f775-503e-4110-9747-87165a60ad3d-tls-cert\") pod \"authorino-5d6965647f-c5b7z\" (UID: \"6904f775-503e-4110-9747-87165a60ad3d\") " pod="kuadrant-system/authorino-5d6965647f-c5b7z" Sep 19 05:53:04.495204 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:04.495174 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6904f775-503e-4110-9747-87165a60ad3d-tls-cert\") pod \"authorino-5d6965647f-c5b7z\" (UID: \"6904f775-503e-4110-9747-87165a60ad3d\") " pod="kuadrant-system/authorino-5d6965647f-c5b7z" Sep 19 05:53:04.500999 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:04.500981 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj4m4\" (UniqueName: \"kubernetes.io/projected/6904f775-503e-4110-9747-87165a60ad3d-kube-api-access-cj4m4\") pod \"authorino-5d6965647f-c5b7z\" (UID: \"6904f775-503e-4110-9747-87165a60ad3d\") " pod="kuadrant-system/authorino-5d6965647f-c5b7z" Sep 19 05:53:04.634151 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:04.634116 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d6965647f-c5b7z" Sep 19 05:53:04.903766 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:04.903741 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d6965647f-c5b7z"] Sep 19 05:53:04.905970 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:53:04.905945 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6904f775_503e_4110_9747_87165a60ad3d.slice/crio-920f8f5c0125988bfa44f9f4ba8f45d5752f504762b2e0c3d2f8f6c3e70df7a1 WatchSource:0}: Error finding container 920f8f5c0125988bfa44f9f4ba8f45d5752f504762b2e0c3d2f8f6c3e70df7a1: Status 404 returned error can't find the container with id 920f8f5c0125988bfa44f9f4ba8f45d5752f504762b2e0c3d2f8f6c3e70df7a1 Sep 19 05:53:05.807791 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:05.807750 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d6965647f-c5b7z" event={"ID":"6904f775-503e-4110-9747-87165a60ad3d","Type":"ContainerStarted","Data":"920f8f5c0125988bfa44f9f4ba8f45d5752f504762b2e0c3d2f8f6c3e70df7a1"} Sep 19 05:53:06.816648 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:06.816606 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d6965647f-c5b7z" event={"ID":"6904f775-503e-4110-9747-87165a60ad3d","Type":"ContainerStarted","Data":"3854cba834693278ac96c7a7dadcd74b44767ac4c273ffd811fb49a0baa70f2e"} Sep 19 05:53:06.832845 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:06.832792 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d6965647f-c5b7z" podStartSLOduration=1.74000616 podStartE2EDuration="2.83277399s" totalImagesPullingTime="1.09276783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:06.831548504 +0000 UTC m=+714.824776048" watchObservedRunningTime="2026-09-19 05:53:06.83277399 +0000 UTC m=+714.826001520" Sep 19 05:53:06.860554 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:06.860515 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c4b6f868b-qwrdp"] Sep 19 05:53:06.861093 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:06.861061 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" podUID="749384e6-4ccf-41b9-8e4f-bfa807a7aab5" containerName="authorino" containerID="cri-o://8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3" gracePeriod=30 Sep 19 05:53:07.133858 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.133829 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" Sep 19 05:53:07.221915 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.221881 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/749384e6-4ccf-41b9-8e4f-bfa807a7aab5-tls-cert\") pod \"749384e6-4ccf-41b9-8e4f-bfa807a7aab5\" (UID: \"749384e6-4ccf-41b9-8e4f-bfa807a7aab5\") " Sep 19 05:53:07.221915 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.221924 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q72s\" (UniqueName: \"kubernetes.io/projected/749384e6-4ccf-41b9-8e4f-bfa807a7aab5-kube-api-access-4q72s\") pod \"749384e6-4ccf-41b9-8e4f-bfa807a7aab5\" (UID: \"749384e6-4ccf-41b9-8e4f-bfa807a7aab5\") " Sep 19 05:53:07.224081 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.224038 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/749384e6-4ccf-41b9-8e4f-bfa807a7aab5-kube-api-access-4q72s" (OuterVolumeSpecName: "kube-api-access-4q72s") pod "749384e6-4ccf-41b9-8e4f-bfa807a7aab5" (UID: "749384e6-4ccf-41b9-8e4f-bfa807a7aab5"). InnerVolumeSpecName "kube-api-access-4q72s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:53:07.234255 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.234216 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/749384e6-4ccf-41b9-8e4f-bfa807a7aab5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "749384e6-4ccf-41b9-8e4f-bfa807a7aab5" (UID: "749384e6-4ccf-41b9-8e4f-bfa807a7aab5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 05:53:07.323453 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.323420 2584 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/749384e6-4ccf-41b9-8e4f-bfa807a7aab5-tls-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:53:07.323453 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.323452 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q72s\" (UniqueName: \"kubernetes.io/projected/749384e6-4ccf-41b9-8e4f-bfa807a7aab5-kube-api-access-4q72s\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:53:07.400676 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.400579 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:07.499887 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.499852 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:07.824574 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.824473 2584 generic.go:356] "Generic (PLEG): container finished" podID="749384e6-4ccf-41b9-8e4f-bfa807a7aab5" containerID="8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3" exitCode=0 Sep 19 05:53:07.824574 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.824535 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" Sep 19 05:53:07.824574 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.824559 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" event={"ID":"749384e6-4ccf-41b9-8e4f-bfa807a7aab5","Type":"ContainerDied","Data":"8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3"} Sep 19 05:53:07.825189 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.824607 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c4b6f868b-qwrdp" event={"ID":"749384e6-4ccf-41b9-8e4f-bfa807a7aab5","Type":"ContainerDied","Data":"9349e484405755b24916cbf74d19771445385e2eb7dfc7d54cb6ce93e261dfaf"} Sep 19 05:53:07.825189 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.824629 2584 scope.go:117] "RemoveContainer" containerID="8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3" Sep 19 05:53:07.835317 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.835292 2584 scope.go:117] "RemoveContainer" containerID="8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3" Sep 19 05:53:07.835713 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:53:07.835682 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3\": container with ID starting with 8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3 not found: ID does not exist" containerID="8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3" Sep 19 05:53:07.835794 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.835714 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3"} err="failed to get container status \"8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3\": rpc error: code = NotFound desc = could not find container \"8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3\": container with ID starting with 8b714fd0967dfc3ad88f1e5f636c8a9afcfc4dddc62bcb5d2e13d44c47742ad3 not found: ID does not exist" Sep 19 05:53:07.848424 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.848372 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c4b6f868b-qwrdp"] Sep 19 05:53:07.852320 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:07.852292 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c4b6f868b-qwrdp"] Sep 19 05:53:08.266487 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:08.266451 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="749384e6-4ccf-41b9-8e4f-bfa807a7aab5" path="/var/lib/kubelet/pods/749384e6-4ccf-41b9-8e4f-bfa807a7aab5/volumes" Sep 19 05:53:08.591014 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:08.590989 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:08.701573 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:08.701546 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:18.362300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:18.362258 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:24.588882 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:24.588843 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:24.689960 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:24.689913 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:24.807554 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:24.807519 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:25.001913 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.001814 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:25.034056 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.034022 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r"] Sep 19 05:53:25.034590 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.034566 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749384e6-4ccf-41b9-8e4f-bfa807a7aab5" containerName="authorino" Sep 19 05:53:25.034590 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.034593 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="749384e6-4ccf-41b9-8e4f-bfa807a7aab5" containerName="authorino" Sep 19 05:53:25.034772 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.034668 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="749384e6-4ccf-41b9-8e4f-bfa807a7aab5" containerName="authorino" Sep 19 05:53:25.040126 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.040103 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.042804 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.042755 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-cc0e26-openshift-default-dockercfg-l65lb\"" Sep 19 05:53:25.042804 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.042771 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 19 05:53:25.049129 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.048952 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r"] Sep 19 05:53:25.105739 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.105702 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:25.197042 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.197008 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:25.203408 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.203372 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d772db8e-bc76-4627-862c-f7588ae39638-istiod-ca-cert\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.203526 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.203442 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d772db8e-bc76-4627-862c-f7588ae39638-istio-podinfo\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.203526 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.203512 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-workload-certs\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.203633 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.203569 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w5n4\" (UniqueName: \"kubernetes.io/projected/d772db8e-bc76-4627-862c-f7588ae39638-kube-api-access-2w5n4\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.203678 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.203641 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-credential-socket\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.203714 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.203691 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-istio-envoy\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.203754 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.203740 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-workload-socket\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.203799 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.203782 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d772db8e-bc76-4627-862c-f7588ae39638-istio-token\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.203843 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.203800 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-istio-data\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.300496 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.300384 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:25.304698 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.304673 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d772db8e-bc76-4627-862c-f7588ae39638-istio-token\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.304799 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.304709 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-istio-data\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.304799 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.304729 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d772db8e-bc76-4627-862c-f7588ae39638-istiod-ca-cert\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.304799 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.304754 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d772db8e-bc76-4627-862c-f7588ae39638-istio-podinfo\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.304799 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.304785 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-workload-certs\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.304996 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.304832 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w5n4\" (UniqueName: \"kubernetes.io/projected/d772db8e-bc76-4627-862c-f7588ae39638-kube-api-access-2w5n4\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.304996 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.304874 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-credential-socket\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.304996 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.304925 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-istio-envoy\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.305160 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.305006 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-workload-socket\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.305217 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.305161 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-istio-data\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.305217 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.305203 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-workload-certs\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.305336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.305297 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-credential-socket\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.305483 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.305462 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-workload-socket\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.305631 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.305609 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d772db8e-bc76-4627-862c-f7588ae39638-istiod-ca-cert\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.307233 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.307212 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-istio-envoy\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.307527 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.307506 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d772db8e-bc76-4627-862c-f7588ae39638-istio-podinfo\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.315050 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.315023 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d772db8e-bc76-4627-862c-f7588ae39638-istio-token\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.315198 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.315180 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w5n4\" (UniqueName: \"kubernetes.io/projected/d772db8e-bc76-4627-862c-f7588ae39638-kube-api-access-2w5n4\") pod \"e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.353228 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.353190 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:25.403022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.402791 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:25.500752 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.500722 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:25.633097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.633073 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r"] Sep 19 05:53:25.635366 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:53:25.635343 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd772db8e_bc76_4627_862c_f7588ae39638.slice/crio-d7285a937ec02121cef885d53c90b92bbebbaefabf40a41ebb6da0555c3b48f5 WatchSource:0}: Error finding container d7285a937ec02121cef885d53c90b92bbebbaefabf40a41ebb6da0555c3b48f5: Status 404 returned error can't find the container with id d7285a937ec02121cef885d53c90b92bbebbaefabf40a41ebb6da0555c3b48f5 Sep 19 05:53:25.637539 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.637519 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 05:53:25.920242 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:25.917550 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" event={"ID":"d772db8e-bc76-4627-862c-f7588ae39638","Type":"ContainerStarted","Data":"d7285a937ec02121cef885d53c90b92bbebbaefabf40a41ebb6da0555c3b48f5"} Sep 19 05:53:26.095958 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:26.095915 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:26.195209 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:26.195114 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:26.570125 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:26.570039 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:28.305488 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:28.305441 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:53:28.305793 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:28.305520 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:53:28.305793 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:28.305549 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:53:28.361337 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:28.361304 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:28.461972 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:28.461931 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:28.597547 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:28.597506 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:28.689613 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:28.689583 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:28.938040 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:28.937945 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" event={"ID":"d772db8e-bc76-4627-862c-f7588ae39638","Type":"ContainerStarted","Data":"60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a"} Sep 19 05:53:28.957716 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:28.957660 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" podStartSLOduration=1.290289258 podStartE2EDuration="3.957643414s" totalImagesPullingTime="2.667354156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:28.956281452 +0000 UTC m=+736.949509006" watchObservedRunningTime="2026-09-19 05:53:28.957643414 +0000 UTC m=+736.950870945" Sep 19 05:53:29.354018 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:29.353984 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:29.359692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:29.359666 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:29.935520 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:29.935488 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:29.942533 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:29.942503 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:29.943452 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:29.943430 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:30.045741 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.045710 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:30.208525 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.208432 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:30.308751 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.308715 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:30.719434 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.719391 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:30.818576 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.818542 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:30.906510 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.906474 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7"] Sep 19 05:53:30.908204 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.908181 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:30.910262 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.910245 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" Sep 19 05:53:30.912818 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.912791 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-3182b5\"" Sep 19 05:53:30.912818 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.912800 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-3182b5-dockercfg-l2jbm\"" Sep 19 05:53:30.920852 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.920827 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7"] Sep 19 05:53:30.942900 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.942867 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf"] Sep 19 05:53:30.947108 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.947085 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" Sep 19 05:53:30.955001 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.954970 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf"] Sep 19 05:53:30.960948 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.960918 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a92660ef-fcfa-427d-98c1-f042db44e380-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7\" (UID: \"a92660ef-fcfa-427d-98c1-f042db44e380\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" Sep 19 05:53:30.961086 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.960967 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rqlh\" (UniqueName: \"kubernetes.io/projected/a92660ef-fcfa-427d-98c1-f042db44e380-kube-api-access-2rqlh\") pod \"payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7\" (UID: \"a92660ef-fcfa-427d-98c1-f042db44e380\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" Sep 19 05:53:30.961270 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.961245 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f2vr\" (UniqueName: \"kubernetes.io/projected/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b-kube-api-access-4f2vr\") pod \"payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf\" (UID: \"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" Sep 19 05:53:30.961358 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:30.961308 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf\" (UID: \"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" Sep 19 05:53:31.004792 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.004698 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:31.062165 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.062128 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f2vr\" (UniqueName: \"kubernetes.io/projected/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b-kube-api-access-4f2vr\") pod \"payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf\" (UID: \"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" Sep 19 05:53:31.062336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.062178 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf\" (UID: \"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" Sep 19 05:53:31.062336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.062205 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a92660ef-fcfa-427d-98c1-f042db44e380-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7\" (UID: \"a92660ef-fcfa-427d-98c1-f042db44e380\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" Sep 19 05:53:31.062336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.062228 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rqlh\" (UniqueName: \"kubernetes.io/projected/a92660ef-fcfa-427d-98c1-f042db44e380-kube-api-access-2rqlh\") pod \"payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7\" (UID: \"a92660ef-fcfa-427d-98c1-f042db44e380\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" Sep 19 05:53:31.063190 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.063166 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf\" (UID: \"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" Sep 19 05:53:31.063229 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.063166 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a92660ef-fcfa-427d-98c1-f042db44e380-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7\" (UID: \"a92660ef-fcfa-427d-98c1-f042db44e380\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" Sep 19 05:53:31.071129 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.071097 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rqlh\" (UniqueName: \"kubernetes.io/projected/a92660ef-fcfa-427d-98c1-f042db44e380-kube-api-access-2rqlh\") pod \"payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7\" (UID: \"a92660ef-fcfa-427d-98c1-f042db44e380\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" Sep 19 05:53:31.071129 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.071109 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f2vr\" (UniqueName: \"kubernetes.io/projected/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b-kube-api-access-4f2vr\") pod \"payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf\" (UID: \"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" Sep 19 05:53:31.104287 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.104256 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:31.221925 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.221883 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" Sep 19 05:53:31.261328 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.261238 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" Sep 19 05:53:31.391168 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.391142 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:31.455586 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.455540 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:31.499032 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.498980 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7"] Sep 19 05:53:31.500579 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:53:31.500550 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda92660ef_fcfa_427d_98c1_f042db44e380.slice/crio-bdb3430eb18bd730ce8673c032db17b1f0ef4b975ae4ebe128cfa643afdf3b81 WatchSource:0}: Error finding container bdb3430eb18bd730ce8673c032db17b1f0ef4b975ae4ebe128cfa643afdf3b81: Status 404 returned error can't find the container with id bdb3430eb18bd730ce8673c032db17b1f0ef4b975ae4ebe128cfa643afdf3b81 Sep 19 05:53:31.546755 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.546725 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf"] Sep 19 05:53:31.549485 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:53:31.549459 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24e5ff24_41b0_4fb4_a2c6_f46774d8f55b.slice/crio-9146bbf80d5a6095dbbfaf545e8512dcac713a5338979589eec1ea326c969aa1 WatchSource:0}: Error finding container 9146bbf80d5a6095dbbfaf545e8512dcac713a5338979589eec1ea326c969aa1: Status 404 returned error can't find the container with id 9146bbf80d5a6095dbbfaf545e8512dcac713a5338979589eec1ea326c969aa1 Sep 19 05:53:31.638732 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.638693 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:31.955704 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.955654 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" event={"ID":"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b","Type":"ContainerStarted","Data":"436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d"} Sep 19 05:53:31.956148 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.955730 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" Sep 19 05:53:31.956148 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.955755 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" event={"ID":"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b","Type":"ContainerStarted","Data":"9146bbf80d5a6095dbbfaf545e8512dcac713a5338979589eec1ea326c969aa1"} Sep 19 05:53:31.957487 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.957458 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" event={"ID":"a92660ef-fcfa-427d-98c1-f042db44e380","Type":"ContainerStarted","Data":"04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f"} Sep 19 05:53:31.957603 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.957504 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" event={"ID":"a92660ef-fcfa-427d-98c1-f042db44e380","Type":"ContainerStarted","Data":"bdb3430eb18bd730ce8673c032db17b1f0ef4b975ae4ebe128cfa643afdf3b81"} Sep 19 05:53:31.957686 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.957667 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" Sep 19 05:53:31.974751 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.974696 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" podStartSLOduration=1.97468027 podStartE2EDuration="1.97468027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:31.973013663 +0000 UTC m=+739.966241188" watchObservedRunningTime="2026-09-19 05:53:31.97468027 +0000 UTC m=+739.967907795" Sep 19 05:53:31.998559 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:31.998481 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" podStartSLOduration=1.99845879 podStartE2EDuration="1.99845879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:31.994667117 +0000 UTC m=+739.987894649" watchObservedRunningTime="2026-09-19 05:53:31.99845879 +0000 UTC m=+739.991686322" Sep 19 05:53:32.042497 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.042456 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:32.147717 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.147688 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:32.154718 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.154679 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w"] Sep 19 05:53:32.158615 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.158591 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" Sep 19 05:53:32.160876 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.160850 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-4b78e2-dockercfg-2g76h\"" Sep 19 05:53:32.161036 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.161019 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-4b78e2\"" Sep 19 05:53:32.166799 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.166490 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk"] Sep 19 05:53:32.170532 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.170506 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" Sep 19 05:53:32.172584 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.172555 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w"] Sep 19 05:53:32.174261 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.174236 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b6d8adb-bd30-41f5-98a5-9dce74f1404f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w\" (UID: \"6b6d8adb-bd30-41f5-98a5-9dce74f1404f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" Sep 19 05:53:32.174375 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.174293 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdrtv\" (UniqueName: \"kubernetes.io/projected/6b6d8adb-bd30-41f5-98a5-9dce74f1404f-kube-api-access-hdrtv\") pod \"payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w\" (UID: \"6b6d8adb-bd30-41f5-98a5-9dce74f1404f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" Sep 19 05:53:32.180338 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.180309 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk"] Sep 19 05:53:32.275556 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.275456 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b6d8adb-bd30-41f5-98a5-9dce74f1404f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w\" (UID: \"6b6d8adb-bd30-41f5-98a5-9dce74f1404f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" Sep 19 05:53:32.275556 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.275517 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdrtv\" (UniqueName: \"kubernetes.io/projected/6b6d8adb-bd30-41f5-98a5-9dce74f1404f-kube-api-access-hdrtv\") pod \"payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w\" (UID: \"6b6d8adb-bd30-41f5-98a5-9dce74f1404f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" Sep 19 05:53:32.275787 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.275567 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk\" (UID: \"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" Sep 19 05:53:32.275787 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.275663 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf6jg\" (UniqueName: \"kubernetes.io/projected/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55-kube-api-access-tf6jg\") pod \"payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk\" (UID: \"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" Sep 19 05:53:32.276641 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.276613 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b6d8adb-bd30-41f5-98a5-9dce74f1404f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w\" (UID: \"6b6d8adb-bd30-41f5-98a5-9dce74f1404f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" Sep 19 05:53:32.283924 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.283902 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdrtv\" (UniqueName: \"kubernetes.io/projected/6b6d8adb-bd30-41f5-98a5-9dce74f1404f-kube-api-access-hdrtv\") pod \"payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w\" (UID: \"6b6d8adb-bd30-41f5-98a5-9dce74f1404f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" Sep 19 05:53:32.376652 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.376621 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf6jg\" (UniqueName: \"kubernetes.io/projected/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55-kube-api-access-tf6jg\") pod \"payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk\" (UID: \"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" Sep 19 05:53:32.376837 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.376702 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk\" (UID: \"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" Sep 19 05:53:32.377371 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.377353 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk\" (UID: \"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" Sep 19 05:53:32.384598 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.384573 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf6jg\" (UniqueName: \"kubernetes.io/projected/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55-kube-api-access-tf6jg\") pod \"payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk\" (UID: \"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" Sep 19 05:53:32.472143 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.472098 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" Sep 19 05:53:32.484474 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.484427 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" Sep 19 05:53:32.649003 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.648970 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:32.787597 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.787574 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w"] Sep 19 05:53:32.789464 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:53:32.789438 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b6d8adb_bd30_41f5_98a5_9dce74f1404f.slice/crio-4e14f055e28bd01537611f08e8a1965e4b8d25d02db6a9772e6c44eb6c657911 WatchSource:0}: Error finding container 4e14f055e28bd01537611f08e8a1965e4b8d25d02db6a9772e6c44eb6c657911: Status 404 returned error can't find the container with id 4e14f055e28bd01537611f08e8a1965e4b8d25d02db6a9772e6c44eb6c657911 Sep 19 05:53:32.813656 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.813630 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk"] Sep 19 05:53:32.821476 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.821451 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:32.827706 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:53:32.827682 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9aeaee4_5cb0_4f12_9ca8_2d5df19fff55.slice/crio-578101d2b40bf8ea96e1f225aff4aa5dd5efaa0c55cc39408f33d70cd4d13c82 WatchSource:0}: Error finding container 578101d2b40bf8ea96e1f225aff4aa5dd5efaa0c55cc39408f33d70cd4d13c82: Status 404 returned error can't find the container with id 578101d2b40bf8ea96e1f225aff4aa5dd5efaa0c55cc39408f33d70cd4d13c82 Sep 19 05:53:32.965230 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.965186 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" event={"ID":"6b6d8adb-bd30-41f5-98a5-9dce74f1404f","Type":"ContainerStarted","Data":"b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e"} Sep 19 05:53:32.965705 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.965370 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" Sep 19 05:53:32.965705 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.965443 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" event={"ID":"6b6d8adb-bd30-41f5-98a5-9dce74f1404f","Type":"ContainerStarted","Data":"4e14f055e28bd01537611f08e8a1965e4b8d25d02db6a9772e6c44eb6c657911"} Sep 19 05:53:32.967147 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.967115 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" event={"ID":"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55","Type":"ContainerStarted","Data":"77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399"} Sep 19 05:53:32.967274 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.967177 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" event={"ID":"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55","Type":"ContainerStarted","Data":"578101d2b40bf8ea96e1f225aff4aa5dd5efaa0c55cc39408f33d70cd4d13c82"} Sep 19 05:53:32.967464 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.967444 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" Sep 19 05:53:32.987923 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:32.987870 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" podStartSLOduration=0.987851624 podStartE2EDuration="987.851624ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:32.98698037 +0000 UTC m=+740.980207900" watchObservedRunningTime="2026-09-19 05:53:32.987851624 +0000 UTC m=+740.981079157" Sep 19 05:53:33.014277 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.014208 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" podStartSLOduration=1.014186056 podStartE2EDuration="1.014186056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:33.010477741 +0000 UTC m=+741.003705275" watchObservedRunningTime="2026-09-19 05:53:33.014186056 +0000 UTC m=+741.007413591" Sep 19 05:53:33.024832 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.024791 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:33.263248 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.263211 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:33.383805 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.383759 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:33.472571 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.472534 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl"] Sep 19 05:53:33.476339 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.476317 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" Sep 19 05:53:33.478825 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.478795 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:33.479391 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.479376 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-cc0e26\"" Sep 19 05:53:33.479691 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.479677 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-cc0e26-dockercfg-vr9qz\"" Sep 19 05:53:33.487642 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.487614 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccafd6f4-5e73-428b-9b15-6770ac1c9b78-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl\" (UID: \"ccafd6f4-5e73-428b-9b15-6770ac1c9b78\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" Sep 19 05:53:33.487733 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.487687 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4jzr\" (UniqueName: \"kubernetes.io/projected/ccafd6f4-5e73-428b-9b15-6770ac1c9b78-kube-api-access-g4jzr\") pod \"payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl\" (UID: \"ccafd6f4-5e73-428b-9b15-6770ac1c9b78\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" Sep 19 05:53:33.490184 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.490161 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl"] Sep 19 05:53:33.552119 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.552030 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz"] Sep 19 05:53:33.555994 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.555970 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" Sep 19 05:53:33.571185 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.571156 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz"] Sep 19 05:53:33.589025 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.588936 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sngt\" (UniqueName: \"kubernetes.io/projected/ebe38f8a-089b-4545-8004-ee46fa95fd1f-kube-api-access-7sngt\") pod \"payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz\" (UID: \"ebe38f8a-089b-4545-8004-ee46fa95fd1f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" Sep 19 05:53:33.589025 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.589018 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe38f8a-089b-4545-8004-ee46fa95fd1f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz\" (UID: \"ebe38f8a-089b-4545-8004-ee46fa95fd1f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" Sep 19 05:53:33.589333 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.589122 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccafd6f4-5e73-428b-9b15-6770ac1c9b78-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl\" (UID: \"ccafd6f4-5e73-428b-9b15-6770ac1c9b78\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" Sep 19 05:53:33.589333 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.589165 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4jzr\" (UniqueName: \"kubernetes.io/projected/ccafd6f4-5e73-428b-9b15-6770ac1c9b78-kube-api-access-g4jzr\") pod \"payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl\" (UID: \"ccafd6f4-5e73-428b-9b15-6770ac1c9b78\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" Sep 19 05:53:33.590030 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.590001 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccafd6f4-5e73-428b-9b15-6770ac1c9b78-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl\" (UID: \"ccafd6f4-5e73-428b-9b15-6770ac1c9b78\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" Sep 19 05:53:33.600416 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.600376 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4jzr\" (UniqueName: \"kubernetes.io/projected/ccafd6f4-5e73-428b-9b15-6770ac1c9b78-kube-api-access-g4jzr\") pod \"payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl\" (UID: \"ccafd6f4-5e73-428b-9b15-6770ac1c9b78\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" Sep 19 05:53:33.690661 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.690622 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sngt\" (UniqueName: \"kubernetes.io/projected/ebe38f8a-089b-4545-8004-ee46fa95fd1f-kube-api-access-7sngt\") pod \"payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz\" (UID: \"ebe38f8a-089b-4545-8004-ee46fa95fd1f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" Sep 19 05:53:33.690834 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.690676 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe38f8a-089b-4545-8004-ee46fa95fd1f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz\" (UID: \"ebe38f8a-089b-4545-8004-ee46fa95fd1f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" Sep 19 05:53:33.691424 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.691379 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe38f8a-089b-4545-8004-ee46fa95fd1f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz\" (UID: \"ebe38f8a-089b-4545-8004-ee46fa95fd1f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" Sep 19 05:53:33.699467 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.699445 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sngt\" (UniqueName: \"kubernetes.io/projected/ebe38f8a-089b-4545-8004-ee46fa95fd1f-kube-api-access-7sngt\") pod \"payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz\" (UID: \"ebe38f8a-089b-4545-8004-ee46fa95fd1f\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" Sep 19 05:53:33.788345 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.788312 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" Sep 19 05:53:33.856352 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.856267 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:33.866797 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.866739 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" Sep 19 05:53:33.972330 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:33.972282 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:34.071100 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:34.071075 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:34.072975 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:34.072951 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl"] Sep 19 05:53:34.074304 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:53:34.074277 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccafd6f4_5e73_428b_9b15_6770ac1c9b78.slice/crio-8fb437603f7a6605e42515d069bda4fe3386434a65aef22e8f339bc854aeab2f WatchSource:0}: Error finding container 8fb437603f7a6605e42515d069bda4fe3386434a65aef22e8f339bc854aeab2f: Status 404 returned error can't find the container with id 8fb437603f7a6605e42515d069bda4fe3386434a65aef22e8f339bc854aeab2f Sep 19 05:53:34.171085 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:34.163603 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz"] Sep 19 05:53:34.196635 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:34.196605 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:34.563376 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:34.563283 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:34.937257 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:34.937224 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:34.980533 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:34.980484 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" event={"ID":"ebe38f8a-089b-4545-8004-ee46fa95fd1f","Type":"ContainerStarted","Data":"45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5"} Sep 19 05:53:34.981011 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:34.980561 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" event={"ID":"ebe38f8a-089b-4545-8004-ee46fa95fd1f","Type":"ContainerStarted","Data":"277d26153d3d561c3701f03713bad760c2a418f8daa2ef9706f01f184d707c82"} Sep 19 05:53:34.981011 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:34.980589 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" Sep 19 05:53:34.986117 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:34.985903 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" event={"ID":"ccafd6f4-5e73-428b-9b15-6770ac1c9b78","Type":"ContainerStarted","Data":"ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad"} Sep 19 05:53:34.986117 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:34.985971 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" event={"ID":"ccafd6f4-5e73-428b-9b15-6770ac1c9b78","Type":"ContainerStarted","Data":"8fb437603f7a6605e42515d069bda4fe3386434a65aef22e8f339bc854aeab2f"} Sep 19 05:53:34.986465 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:34.986440 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" Sep 19 05:53:35.007237 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:35.007184 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" podStartSLOduration=2.007164605 podStartE2EDuration="2.007164605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:35.005815241 +0000 UTC m=+742.999042798" watchObservedRunningTime="2026-09-19 05:53:35.007164605 +0000 UTC m=+743.000392140" Sep 19 05:53:35.026120 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:35.026067 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" podStartSLOduration=2.026047478 podStartE2EDuration="2.026047478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:35.025674703 +0000 UTC m=+743.018902236" watchObservedRunningTime="2026-09-19 05:53:35.026047478 +0000 UTC m=+743.019275013" Sep 19 05:53:35.057548 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:35.057524 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:35.698828 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:35.698798 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:36.592210 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:36.592178 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:36.816769 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:36.816738 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:36.916763 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:36.916676 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:37.499913 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:37.499882 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:37.775954 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:37.775862 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:38.553341 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:38.553307 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:38.669308 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:38.669277 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:38.804945 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:38.804861 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:38.907333 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:38.907301 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:39.004661 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.004632 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:39.242886 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.242838 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:39.340130 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.340097 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:39.431289 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.431246 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm"] Sep 19 05:53:39.435838 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.435813 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" Sep 19 05:53:39.438227 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.438201 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-bd4c1118\"" Sep 19 05:53:39.438352 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.438310 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-bd4c1118-dockercfg-dx26n\"" Sep 19 05:53:39.444691 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.444379 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm"] Sep 19 05:53:39.552113 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.552013 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk8dc\" (UniqueName: \"kubernetes.io/projected/428cb3b9-1e1c-48d0-be9c-283dba34fef7-kube-api-access-lk8dc\") pod \"payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm\" (UID: \"428cb3b9-1e1c-48d0-be9c-283dba34fef7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" Sep 19 05:53:39.552275 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.552168 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428cb3b9-1e1c-48d0-be9c-283dba34fef7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm\" (UID: \"428cb3b9-1e1c-48d0-be9c-283dba34fef7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" Sep 19 05:53:39.653212 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.653170 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428cb3b9-1e1c-48d0-be9c-283dba34fef7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm\" (UID: \"428cb3b9-1e1c-48d0-be9c-283dba34fef7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" Sep 19 05:53:39.653385 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.653281 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk8dc\" (UniqueName: \"kubernetes.io/projected/428cb3b9-1e1c-48d0-be9c-283dba34fef7-kube-api-access-lk8dc\") pod \"payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm\" (UID: \"428cb3b9-1e1c-48d0-be9c-283dba34fef7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" Sep 19 05:53:39.653914 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.653891 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428cb3b9-1e1c-48d0-be9c-283dba34fef7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm\" (UID: \"428cb3b9-1e1c-48d0-be9c-283dba34fef7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" Sep 19 05:53:39.661931 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.661906 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk8dc\" (UniqueName: \"kubernetes.io/projected/428cb3b9-1e1c-48d0-be9c-283dba34fef7-kube-api-access-lk8dc\") pod \"payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm\" (UID: \"428cb3b9-1e1c-48d0-be9c-283dba34fef7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" Sep 19 05:53:39.747625 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.747593 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" Sep 19 05:53:39.816624 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:39.816583 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:40.051142 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:40.051115 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm"] Sep 19 05:53:40.053198 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:53:40.053166 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod428cb3b9_1e1c_48d0_be9c_283dba34fef7.slice/crio-3f80518397fec984e07784df1fdbf49f8411b0ab3170f2e296cb924ecb9be3fa WatchSource:0}: Error finding container 3f80518397fec984e07784df1fdbf49f8411b0ab3170f2e296cb924ecb9be3fa: Status 404 returned error can't find the container with id 3f80518397fec984e07784df1fdbf49f8411b0ab3170f2e296cb924ecb9be3fa Sep 19 05:53:40.736212 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:40.736182 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:41.032088 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:41.031976 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" event={"ID":"428cb3b9-1e1c-48d0-be9c-283dba34fef7","Type":"ContainerStarted","Data":"f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468"} Sep 19 05:53:41.032088 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:41.032059 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" event={"ID":"428cb3b9-1e1c-48d0-be9c-283dba34fef7","Type":"ContainerStarted","Data":"3f80518397fec984e07784df1fdbf49f8411b0ab3170f2e296cb924ecb9be3fa"} Sep 19 05:53:41.033431 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:41.032930 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" Sep 19 05:53:41.048454 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:41.048378 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" podStartSLOduration=2.048362568 podStartE2EDuration="2.048362568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:41.047935072 +0000 UTC m=+749.041162598" watchObservedRunningTime="2026-09-19 05:53:41.048362568 +0000 UTC m=+749.041590099" Sep 19 05:53:41.618275 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:41.618245 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:42.670490 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:42.670462 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:42.776874 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:42.776817 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:42.969363 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:42.969279 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" Sep 19 05:53:42.969363 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:42.969356 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" Sep 19 05:53:43.129150 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.129120 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:43.242265 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.242168 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:43.539957 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.539856 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:43.640705 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.640673 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:43.680120 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.680078 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f"] Sep 19 05:53:43.690985 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.690946 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" Sep 19 05:53:43.693818 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.693794 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-8128f3\"" Sep 19 05:53:43.698931 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.698905 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f"] Sep 19 05:53:43.745701 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.745657 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:43.795747 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.795658 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03ab01be-ae18-4ef2-91e0-b8c21222d49e-plugins-config-volume\") pod \"payload-processing-e2e-autores-8128f3-7b47db56fc-5425f\" (UID: \"03ab01be-ae18-4ef2-91e0-b8c21222d49e\") " pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" Sep 19 05:53:43.795899 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.795806 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpzkd\" (UniqueName: \"kubernetes.io/projected/03ab01be-ae18-4ef2-91e0-b8c21222d49e-kube-api-access-kpzkd\") pod \"payload-processing-e2e-autores-8128f3-7b47db56fc-5425f\" (UID: \"03ab01be-ae18-4ef2-91e0-b8c21222d49e\") " pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" Sep 19 05:53:43.897244 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.897199 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03ab01be-ae18-4ef2-91e0-b8c21222d49e-plugins-config-volume\") pod \"payload-processing-e2e-autores-8128f3-7b47db56fc-5425f\" (UID: \"03ab01be-ae18-4ef2-91e0-b8c21222d49e\") " pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" Sep 19 05:53:43.897452 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.897353 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpzkd\" (UniqueName: \"kubernetes.io/projected/03ab01be-ae18-4ef2-91e0-b8c21222d49e-kube-api-access-kpzkd\") pod \"payload-processing-e2e-autores-8128f3-7b47db56fc-5425f\" (UID: \"03ab01be-ae18-4ef2-91e0-b8c21222d49e\") " pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" Sep 19 05:53:43.897999 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.897974 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03ab01be-ae18-4ef2-91e0-b8c21222d49e-plugins-config-volume\") pod \"payload-processing-e2e-autores-8128f3-7b47db56fc-5425f\" (UID: \"03ab01be-ae18-4ef2-91e0-b8c21222d49e\") " pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" Sep 19 05:53:43.906669 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.906643 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpzkd\" (UniqueName: \"kubernetes.io/projected/03ab01be-ae18-4ef2-91e0-b8c21222d49e-kube-api-access-kpzkd\") pod \"payload-processing-e2e-autores-8128f3-7b47db56fc-5425f\" (UID: \"03ab01be-ae18-4ef2-91e0-b8c21222d49e\") " pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" Sep 19 05:53:43.912421 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.910232 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:43.975568 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.975529 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" Sep 19 05:53:43.975741 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:43.975699 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" Sep 19 05:53:44.003584 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:44.003549 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" Sep 19 05:53:44.064372 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:44.064068 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:44.165081 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:44.165042 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:44.325105 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:44.324616 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f"] Sep 19 05:53:44.336393 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:53:44.336357 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03ab01be_ae18_4ef2_91e0_b8c21222d49e.slice/crio-2fcfe20b1b9d98d4ceb23fa4f8208705264a5d2e402175daff87ded54763c0f6 WatchSource:0}: Error finding container 2fcfe20b1b9d98d4ceb23fa4f8208705264a5d2e402175daff87ded54763c0f6: Status 404 returned error can't find the container with id 2fcfe20b1b9d98d4ceb23fa4f8208705264a5d2e402175daff87ded54763c0f6 Sep 19 05:53:45.075151 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:45.075112 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" event={"ID":"03ab01be-ae18-4ef2-91e0-b8c21222d49e","Type":"ContainerStarted","Data":"c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c"} Sep 19 05:53:45.075596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:45.075262 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" Sep 19 05:53:45.075596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:45.075311 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" event={"ID":"03ab01be-ae18-4ef2-91e0-b8c21222d49e","Type":"ContainerStarted","Data":"2fcfe20b1b9d98d4ceb23fa4f8208705264a5d2e402175daff87ded54763c0f6"} Sep 19 05:53:45.098192 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:45.098139 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" podStartSLOduration=2.098119962 podStartE2EDuration="2.098119962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:45.095749198 +0000 UTC m=+753.088976724" watchObservedRunningTime="2026-09-19 05:53:45.098119962 +0000 UTC m=+753.091347492" Sep 19 05:53:45.188101 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:45.188051 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:45.360491 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:45.360460 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:45.523064 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:45.521355 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:45.611933 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:45.611856 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:45.711246 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:45.711213 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:45.993841 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:45.993750 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" Sep 19 05:53:45.994008 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:45.993842 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" Sep 19 05:53:46.108165 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:46.108135 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:47.072710 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:47.072670 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:47.627591 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:47.627554 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:47.728769 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:47.728739 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:48.104976 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:48.104945 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:48.250060 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:48.250035 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:48.459518 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:48.459436 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:48.735293 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:48.735182 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:48.838335 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:48.838281 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:48.933983 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:48.933957 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:49.043953 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:49.043867 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:49.104583 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:49.104548 2584 generic.go:356] "Generic (PLEG): container finished" podID="d772db8e-bc76-4627-862c-f7588ae39638" containerID="60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a" exitCode=0 Sep 19 05:53:49.104782 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:49.104597 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" event={"ID":"d772db8e-bc76-4627-862c-f7588ae39638","Type":"ContainerDied","Data":"60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a"} Sep 19 05:53:49.105192 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:49.105164 2584 scope.go:117] "RemoveContainer" containerID="60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a" Sep 19 05:53:49.106609 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:49.106577 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:53:49.106723 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:49.106668 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:53:49.106787 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:49.106720 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:53:49.942635 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:49.942588 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:50.115737 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:50.115705 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" event={"ID":"d772db8e-bc76-4627-862c-f7588ae39638","Type":"ContainerStarted","Data":"e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082"} Sep 19 05:53:50.307157 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:50.306952 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:50.353712 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:50.353673 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:50.354000 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:50.353974 2584 patch_prober.go:28] interesting pod/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Sep 19 05:53:50.354075 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:50.354028 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Sep 19 05:53:50.775209 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:50.775176 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:50.915333 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:50.914925 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:51.250058 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:51.250015 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:51.338460 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:51.338409 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:51.354621 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:51.354588 2584 patch_prober.go:28] interesting pod/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Sep 19 05:53:51.354763 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:51.354640 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Sep 19 05:53:51.437978 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:51.437949 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:51.898630 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:51.898602 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:52.004596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.004557 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:52.354423 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.354362 2584 patch_prober.go:28] interesting pod/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Sep 19 05:53:52.354806 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.354444 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Sep 19 05:53:52.531966 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.531927 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:52.631486 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.630439 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:52.677686 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.673972 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck"] Sep 19 05:53:52.681443 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.681411 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" Sep 19 05:53:52.683872 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.683840 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-c85316\"" Sep 19 05:53:52.698866 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.698835 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck"] Sep 19 05:53:52.795601 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.795571 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2623fdd9-d3ce-4cde-9983-2434e6b2e240-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck\" (UID: \"2623fdd9-d3ce-4cde-9983-2434e6b2e240\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" Sep 19 05:53:52.795776 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.795619 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2p6h\" (UniqueName: \"kubernetes.io/projected/2623fdd9-d3ce-4cde-9983-2434e6b2e240-kube-api-access-h2p6h\") pod \"payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck\" (UID: \"2623fdd9-d3ce-4cde-9983-2434e6b2e240\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" Sep 19 05:53:52.897185 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.897086 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2623fdd9-d3ce-4cde-9983-2434e6b2e240-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck\" (UID: \"2623fdd9-d3ce-4cde-9983-2434e6b2e240\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" Sep 19 05:53:52.897185 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.897134 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2p6h\" (UniqueName: \"kubernetes.io/projected/2623fdd9-d3ce-4cde-9983-2434e6b2e240-kube-api-access-h2p6h\") pod \"payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck\" (UID: \"2623fdd9-d3ce-4cde-9983-2434e6b2e240\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" Sep 19 05:53:52.897857 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.897828 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2623fdd9-d3ce-4cde-9983-2434e6b2e240-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck\" (UID: \"2623fdd9-d3ce-4cde-9983-2434e6b2e240\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" Sep 19 05:53:52.916136 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.916094 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2p6h\" (UniqueName: \"kubernetes.io/projected/2623fdd9-d3ce-4cde-9983-2434e6b2e240-kube-api-access-h2p6h\") pod \"payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck\" (UID: \"2623fdd9-d3ce-4cde-9983-2434e6b2e240\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" Sep 19 05:53:52.992420 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:52.992367 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" Sep 19 05:53:53.035917 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:53.035886 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:53.047443 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:53.047416 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" Sep 19 05:53:53.137185 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:53.137051 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:53.301036 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:53.301011 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck"] Sep 19 05:53:53.303626 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:53:53.303598 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2623fdd9_d3ce_4cde_9983_2434e6b2e240.slice/crio-5a928e97bc6351662f34ceb6946bb6d33e5d652af8faf8399d8d303ede8c28f6 WatchSource:0}: Error finding container 5a928e97bc6351662f34ceb6946bb6d33e5d652af8faf8399d8d303ede8c28f6: Status 404 returned error can't find the container with id 5a928e97bc6351662f34ceb6946bb6d33e5d652af8faf8399d8d303ede8c28f6 Sep 19 05:53:53.354636 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:53.354611 2584 patch_prober.go:28] interesting pod/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Sep 19 05:53:53.354941 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:53.354674 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Sep 19 05:53:54.106640 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:54.106602 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:54.149288 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:54.149253 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" event={"ID":"2623fdd9-d3ce-4cde-9983-2434e6b2e240","Type":"ContainerStarted","Data":"a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a"} Sep 19 05:53:54.149523 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:54.149319 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" Sep 19 05:53:54.149523 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:54.149337 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" event={"ID":"2623fdd9-d3ce-4cde-9983-2434e6b2e240","Type":"ContainerStarted","Data":"5a928e97bc6351662f34ceb6946bb6d33e5d652af8faf8399d8d303ede8c28f6"} Sep 19 05:53:54.173126 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:54.173072 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" podStartSLOduration=2.173055651 podStartE2EDuration="2.173055651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:53:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:53:54.17079709 +0000 UTC m=+762.164024621" watchObservedRunningTime="2026-09-19 05:53:54.173055651 +0000 UTC m=+762.166283182" Sep 19 05:53:54.354069 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:54.354035 2584 patch_prober.go:28] interesting pod/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Sep 19 05:53:54.354246 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:54.354100 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Sep 19 05:53:55.179948 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:55.179916 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:55.354010 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:55.353974 2584 patch_prober.go:28] interesting pod/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Sep 19 05:53:55.354191 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:55.354031 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Sep 19 05:53:55.882342 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:55.882310 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:56.083238 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:56.083210 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" Sep 19 05:53:56.357430 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:56.357378 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:56.357865 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:56.357705 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:56.358393 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:56.358373 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 05:53:56.705649 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:56.705557 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:57.273989 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:57.273952 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d"] Sep 19 05:53:57.277448 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:57.277388 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d" Sep 19 05:53:57.279775 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:57.279751 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 05:53:57.286503 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:57.286468 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d"] Sep 19 05:53:57.447279 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:57.447238 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b7b5ab8-2303-4861-82b4-e8300437dd14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-bd4c1118-s544d\" (UID: \"7b7b5ab8-2303-4861-82b4-e8300437dd14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d" Sep 19 05:53:57.509455 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:57.509414 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:57.549014 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:57.548916 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b7b5ab8-2303-4861-82b4-e8300437dd14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-bd4c1118-s544d\" (UID: \"7b7b5ab8-2303-4861-82b4-e8300437dd14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d" Sep 19 05:53:57.549591 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:57.549573 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b7b5ab8-2303-4861-82b4-e8300437dd14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-bd4c1118-s544d\" (UID: \"7b7b5ab8-2303-4861-82b4-e8300437dd14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d" Sep 19 05:53:57.589022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:57.588984 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d" Sep 19 05:53:57.863391 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:57.863364 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d"] Sep 19 05:53:57.866079 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:53:57.866037 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b7b5ab8_2303_4861_82b4_e8300437dd14.slice/crio-c6f8f2eaf1c6c0e65f9c51afa9654a13ab2bcb4e430c44998e918e944c90c711 WatchSource:0}: Error finding container c6f8f2eaf1c6c0e65f9c51afa9654a13ab2bcb4e430c44998e918e944c90c711: Status 404 returned error can't find the container with id c6f8f2eaf1c6c0e65f9c51afa9654a13ab2bcb4e430c44998e918e944c90c711 Sep 19 05:53:58.173225 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:58.173130 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d" event={"ID":"7b7b5ab8-2303-4861-82b4-e8300437dd14","Type":"ContainerStarted","Data":"c6f8f2eaf1c6c0e65f9c51afa9654a13ab2bcb4e430c44998e918e944c90c711"} Sep 19 05:53:58.400643 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:58.400610 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:58.671817 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:58.671754 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:53:59.211925 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:53:59.211890 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:00.089987 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:00.089954 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:00.804681 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:00.804653 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:01.191438 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:01.191381 2584 generic.go:356] "Generic (PLEG): container finished" podID="7b7b5ab8-2303-4861-82b4-e8300437dd14" containerID="21b48826f3b22135a1273bd39fb110c7ddb5d860e58f659029bc326b10eaf09b" exitCode=0 Sep 19 05:54:01.191907 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:01.191463 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d" event={"ID":"7b7b5ab8-2303-4861-82b4-e8300437dd14","Type":"ContainerDied","Data":"21b48826f3b22135a1273bd39fb110c7ddb5d860e58f659029bc326b10eaf09b"} Sep 19 05:54:01.487970 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:01.487847 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:02.211802 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:02.211773 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:02.342427 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:02.342375 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d" Sep 19 05:54:02.500896 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:02.500797 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b7b5ab8-2303-4861-82b4-e8300437dd14-maas-api-service-ca\") pod \"7b7b5ab8-2303-4861-82b4-e8300437dd14\" (UID: \"7b7b5ab8-2303-4861-82b4-e8300437dd14\") " Sep 19 05:54:02.501192 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:02.501165 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b7b5ab8-2303-4861-82b4-e8300437dd14-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b7b5ab8-2303-4861-82b4-e8300437dd14" (UID: "7b7b5ab8-2303-4861-82b4-e8300437dd14"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:54:02.601952 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:02.601905 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b7b5ab8-2303-4861-82b4-e8300437dd14-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:54:03.089152 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:03.089112 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:03.204557 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:03.204529 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d" Sep 19 05:54:03.204708 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:03.204529 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d" event={"ID":"7b7b5ab8-2303-4861-82b4-e8300437dd14","Type":"ContainerDied","Data":"c6f8f2eaf1c6c0e65f9c51afa9654a13ab2bcb4e430c44998e918e944c90c711"} Sep 19 05:54:03.204708 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:03.204655 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6f8f2eaf1c6c0e65f9c51afa9654a13ab2bcb4e430c44998e918e944c90c711" Sep 19 05:54:03.912733 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:03.912702 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:05.156911 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:05.156878 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" Sep 19 05:54:05.396759 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:05.396726 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:06.193301 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:06.193271 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:06.893698 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:06.893666 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:07.413707 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:07.413664 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:07.568123 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:07.568091 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:08.243318 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.243286 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:08.278728 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.278696 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm"] Sep 19 05:54:08.278949 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.278920 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" podUID="428cb3b9-1e1c-48d0-be9c-283dba34fef7" containerName="payload-pre-processing" containerID="cri-o://f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468" gracePeriod=30 Sep 19 05:54:08.346559 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.346528 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:08.450583 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.450513 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:08.452695 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.452640 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d"] Sep 19 05:54:08.454982 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.454955 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-bd4c1118-s544d"] Sep 19 05:54:08.521311 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.521285 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:08.552170 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.552139 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" Sep 19 05:54:08.559554 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.559523 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lk8dc\" (UniqueName: \"kubernetes.io/projected/428cb3b9-1e1c-48d0-be9c-283dba34fef7-kube-api-access-lk8dc\") pod \"428cb3b9-1e1c-48d0-be9c-283dba34fef7\" (UID: \"428cb3b9-1e1c-48d0-be9c-283dba34fef7\") " Sep 19 05:54:08.559732 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.559574 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428cb3b9-1e1c-48d0-be9c-283dba34fef7-plugins-config-volume\") pod \"428cb3b9-1e1c-48d0-be9c-283dba34fef7\" (UID: \"428cb3b9-1e1c-48d0-be9c-283dba34fef7\") " Sep 19 05:54:08.560144 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.560110 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/428cb3b9-1e1c-48d0-be9c-283dba34fef7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "428cb3b9-1e1c-48d0-be9c-283dba34fef7" (UID: "428cb3b9-1e1c-48d0-be9c-283dba34fef7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:54:08.561901 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.561875 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/428cb3b9-1e1c-48d0-be9c-283dba34fef7-kube-api-access-lk8dc" (OuterVolumeSpecName: "kube-api-access-lk8dc") pod "428cb3b9-1e1c-48d0-be9c-283dba34fef7" (UID: "428cb3b9-1e1c-48d0-be9c-283dba34fef7"). InnerVolumeSpecName "kube-api-access-lk8dc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:54:08.629483 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.629450 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:08.661022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.660991 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lk8dc\" (UniqueName: \"kubernetes.io/projected/428cb3b9-1e1c-48d0-be9c-283dba34fef7-kube-api-access-lk8dc\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:54:08.661022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:08.661018 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428cb3b9-1e1c-48d0-be9c-283dba34fef7-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:54:09.089795 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.089757 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:09.189234 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.189197 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:09.241371 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.241335 2584 generic.go:356] "Generic (PLEG): container finished" podID="428cb3b9-1e1c-48d0-be9c-283dba34fef7" containerID="f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468" exitCode=0 Sep 19 05:54:09.241575 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.241383 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" Sep 19 05:54:09.241575 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.241427 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" event={"ID":"428cb3b9-1e1c-48d0-be9c-283dba34fef7","Type":"ContainerDied","Data":"f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468"} Sep 19 05:54:09.241575 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.241484 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm" event={"ID":"428cb3b9-1e1c-48d0-be9c-283dba34fef7","Type":"ContainerDied","Data":"3f80518397fec984e07784df1fdbf49f8411b0ab3170f2e296cb924ecb9be3fa"} Sep 19 05:54:09.241575 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.241508 2584 scope.go:117] "RemoveContainer" containerID="f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468" Sep 19 05:54:09.251434 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.251417 2584 scope.go:117] "RemoveContainer" containerID="f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468" Sep 19 05:54:09.251692 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:54:09.251673 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468\": container with ID starting with f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468 not found: ID does not exist" containerID="f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468" Sep 19 05:54:09.251766 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.251700 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468"} err="failed to get container status \"f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468\": rpc error: code = NotFound desc = could not find container \"f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468\": container with ID starting with f8c02184257bec43f1d5af88a1648c5c38110635fdfc1e2238122ef265df9468 not found: ID does not exist" Sep 19 05:54:09.269805 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.269776 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm"] Sep 19 05:54:09.278291 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.278265 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd4c1118-67dd68b8fd-jvhxm"] Sep 19 05:54:09.320173 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.320143 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:09.418432 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.418308 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:09.517359 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.517314 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:09.653740 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:09.653708 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:10.120693 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:10.120652 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:10.224579 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:10.224542 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:10.267719 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:10.267686 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="428cb3b9-1e1c-48d0-be9c-283dba34fef7" path="/var/lib/kubelet/pods/428cb3b9-1e1c-48d0-be9c-283dba34fef7/volumes" Sep 19 05:54:10.268143 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:10.268123 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b7b5ab8-2303-4861-82b4-e8300437dd14" path="/var/lib/kubelet/pods/7b7b5ab8-2303-4861-82b4-e8300437dd14/volumes" Sep 19 05:54:11.058413 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:11.058370 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:11.872765 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:11.872731 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:12.608623 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:12.608589 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:12.705358 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:12.705329 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:13.339671 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:13.339637 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:14.019455 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.019393 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:14.119643 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.119611 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:14.461187 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.461152 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:14.600311 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.600277 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:14.703193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.703159 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:14.704271 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.704251 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd"] Sep 19 05:54:14.704748 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.704735 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="428cb3b9-1e1c-48d0-be9c-283dba34fef7" containerName="payload-pre-processing" Sep 19 05:54:14.704788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.704750 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="428cb3b9-1e1c-48d0-be9c-283dba34fef7" containerName="payload-pre-processing" Sep 19 05:54:14.704788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.704778 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b7b5ab8-2303-4861-82b4-e8300437dd14" containerName="revoke-keys" Sep 19 05:54:14.704788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.704783 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b7b5ab8-2303-4861-82b4-e8300437dd14" containerName="revoke-keys" Sep 19 05:54:14.704886 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.704856 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="428cb3b9-1e1c-48d0-be9c-283dba34fef7" containerName="payload-pre-processing" Sep 19 05:54:14.704886 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.704864 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b7b5ab8-2303-4861-82b4-e8300437dd14" containerName="revoke-keys" Sep 19 05:54:14.709413 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.709384 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" Sep 19 05:54:14.711854 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.711788 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-0c7c7a36\"" Sep 19 05:54:14.711854 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.711793 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-0c7c7a36-dockercfg-ndw2l\"" Sep 19 05:54:14.718690 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.718665 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd"] Sep 19 05:54:14.820810 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.820775 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7ebe0cc-858a-4af3-83c9-8a458547a335-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd\" (UID: \"f7ebe0cc-858a-4af3-83c9-8a458547a335\") " pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" Sep 19 05:54:14.821025 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.820857 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb6lx\" (UniqueName: \"kubernetes.io/projected/f7ebe0cc-858a-4af3-83c9-8a458547a335-kube-api-access-fb6lx\") pod \"payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd\" (UID: \"f7ebe0cc-858a-4af3-83c9-8a458547a335\") " pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" Sep 19 05:54:14.922497 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.922465 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb6lx\" (UniqueName: \"kubernetes.io/projected/f7ebe0cc-858a-4af3-83c9-8a458547a335-kube-api-access-fb6lx\") pod \"payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd\" (UID: \"f7ebe0cc-858a-4af3-83c9-8a458547a335\") " pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" Sep 19 05:54:14.922671 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.922589 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7ebe0cc-858a-4af3-83c9-8a458547a335-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd\" (UID: \"f7ebe0cc-858a-4af3-83c9-8a458547a335\") " pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" Sep 19 05:54:14.923321 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.923300 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7ebe0cc-858a-4af3-83c9-8a458547a335-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd\" (UID: \"f7ebe0cc-858a-4af3-83c9-8a458547a335\") " pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" Sep 19 05:54:14.929794 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:14.929770 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb6lx\" (UniqueName: \"kubernetes.io/projected/f7ebe0cc-858a-4af3-83c9-8a458547a335-kube-api-access-fb6lx\") pod \"payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd\" (UID: \"f7ebe0cc-858a-4af3-83c9-8a458547a335\") " pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" Sep 19 05:54:15.023094 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:15.022979 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" Sep 19 05:54:15.213131 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:15.213085 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:15.299573 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:15.299535 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd"] Sep 19 05:54:15.301147 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:54:15.301122 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ebe0cc_858a_4af3_83c9_8a458547a335.slice/crio-1ba6230aecd50c217459db4c31fac8afdd47ddc7762e5d3c4c3ff7ba31a12eb7 WatchSource:0}: Error finding container 1ba6230aecd50c217459db4c31fac8afdd47ddc7762e5d3c4c3ff7ba31a12eb7: Status 404 returned error can't find the container with id 1ba6230aecd50c217459db4c31fac8afdd47ddc7762e5d3c4c3ff7ba31a12eb7 Sep 19 05:54:16.137644 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:16.137616 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:16.239817 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:16.239786 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:16.283311 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:16.283273 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" event={"ID":"f7ebe0cc-858a-4af3-83c9-8a458547a335","Type":"ContainerStarted","Data":"d40e8754d4b2eb68d5d47304e9bb8600c8b9c6d34ac55e2e507a16b86e66714b"} Sep 19 05:54:16.283495 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:16.283326 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" event={"ID":"f7ebe0cc-858a-4af3-83c9-8a458547a335","Type":"ContainerStarted","Data":"1ba6230aecd50c217459db4c31fac8afdd47ddc7762e5d3c4c3ff7ba31a12eb7"} Sep 19 05:54:16.283495 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:16.283369 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" Sep 19 05:54:16.304754 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:16.304702 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" podStartSLOduration=2.304674883 podStartE2EDuration="2.304674883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:54:16.302485358 +0000 UTC m=+784.295712891" watchObservedRunningTime="2026-09-19 05:54:16.304674883 +0000 UTC m=+784.297902414" Sep 19 05:54:16.312846 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:16.312813 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:16.428156 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:16.428075 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:16.525582 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:16.525535 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:16.674290 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:16.674254 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:17.111787 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:17.111754 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:17.862614 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:17.862584 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:19.137518 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:19.137489 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:22.670772 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:22.670737 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:22.747035 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:22.747008 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:27.292212 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:27.292134 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" Sep 19 05:54:36.855780 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:36.855745 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:38.519366 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:38.519307 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:39.076612 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:39.076574 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:39.685733 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:39.685693 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:40.493774 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:40.493742 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:41.185151 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:41.185104 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:41.814855 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:41.814824 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:42.617080 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:42.617046 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:42.706744 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:42.706707 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4"] Sep 19 05:54:42.713906 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:42.713880 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4" Sep 19 05:54:42.716391 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:42.716355 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 05:54:42.717921 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:42.717884 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4"] Sep 19 05:54:42.815533 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:42.815493 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3f476bd-aa57-46c1-bf0c-95db65296b76-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4\" (UID: \"d3f476bd-aa57-46c1-bf0c-95db65296b76\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4" Sep 19 05:54:42.916222 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:42.916128 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3f476bd-aa57-46c1-bf0c-95db65296b76-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4\" (UID: \"d3f476bd-aa57-46c1-bf0c-95db65296b76\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4" Sep 19 05:54:42.916817 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:42.916799 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3f476bd-aa57-46c1-bf0c-95db65296b76-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4\" (UID: \"d3f476bd-aa57-46c1-bf0c-95db65296b76\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4" Sep 19 05:54:43.027019 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:43.026981 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4" Sep 19 05:54:43.300883 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:43.300851 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4"] Sep 19 05:54:43.303156 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:54:43.303121 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f476bd_aa57_46c1_bf0c_95db65296b76.slice/crio-da014ac96053e9edea77fba883451741220b9f7f9a8733cc22e26a87f8456dc8 WatchSource:0}: Error finding container da014ac96053e9edea77fba883451741220b9f7f9a8733cc22e26a87f8456dc8: Status 404 returned error can't find the container with id da014ac96053e9edea77fba883451741220b9f7f9a8733cc22e26a87f8456dc8 Sep 19 05:54:43.394301 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:43.394268 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:43.438816 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:43.438782 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4" event={"ID":"d3f476bd-aa57-46c1-bf0c-95db65296b76","Type":"ContainerStarted","Data":"da014ac96053e9edea77fba883451741220b9f7f9a8733cc22e26a87f8456dc8"} Sep 19 05:54:44.123887 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:44.123854 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:44.445534 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:44.445497 2584 generic.go:356] "Generic (PLEG): container finished" podID="d3f476bd-aa57-46c1-bf0c-95db65296b76" containerID="4a4661cb96f26d1506891e398a8561eed9355b2cb2e3440fded0c6148d03dc34" exitCode=0 Sep 19 05:54:44.445703 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:44.445543 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4" event={"ID":"d3f476bd-aa57-46c1-bf0c-95db65296b76","Type":"ContainerDied","Data":"4a4661cb96f26d1506891e398a8561eed9355b2cb2e3440fded0c6148d03dc34"} Sep 19 05:54:44.905537 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:44.905507 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:45.589003 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:45.588976 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:45.602202 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:45.602179 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4" Sep 19 05:54:45.647798 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:45.647764 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3f476bd-aa57-46c1-bf0c-95db65296b76-maas-api-service-ca\") pod \"d3f476bd-aa57-46c1-bf0c-95db65296b76\" (UID: \"d3f476bd-aa57-46c1-bf0c-95db65296b76\") " Sep 19 05:54:45.648113 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:45.648091 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3f476bd-aa57-46c1-bf0c-95db65296b76-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d3f476bd-aa57-46c1-bf0c-95db65296b76" (UID: "d3f476bd-aa57-46c1-bf0c-95db65296b76"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:54:45.749242 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:45.749154 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3f476bd-aa57-46c1-bf0c-95db65296b76-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:54:46.300264 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:46.300234 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:46.458030 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:46.457993 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4" event={"ID":"d3f476bd-aa57-46c1-bf0c-95db65296b76","Type":"ContainerDied","Data":"da014ac96053e9edea77fba883451741220b9f7f9a8733cc22e26a87f8456dc8"} Sep 19 05:54:46.458193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:46.458047 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da014ac96053e9edea77fba883451741220b9f7f9a8733cc22e26a87f8456dc8" Sep 19 05:54:46.458193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:46.458005 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4" Sep 19 05:54:46.976870 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:46.976841 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:47.824951 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:47.824918 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:49.416574 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:49.416531 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:50.064454 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:50.064425 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:53.728695 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:53.728660 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:54.335627 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.335598 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:54.356786 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.356743 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd"] Sep 19 05:54:54.357037 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.357005 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" podUID="f7ebe0cc-858a-4af3-83c9-8a458547a335" containerName="payload-processing" containerID="cri-o://d40e8754d4b2eb68d5d47304e9bb8600c8b9c6d34ac55e2e507a16b86e66714b" gracePeriod=30 Sep 19 05:54:54.438389 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.438342 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:54.507010 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.506783 2584 generic.go:356] "Generic (PLEG): container finished" podID="f7ebe0cc-858a-4af3-83c9-8a458547a335" containerID="d40e8754d4b2eb68d5d47304e9bb8600c8b9c6d34ac55e2e507a16b86e66714b" exitCode=0 Sep 19 05:54:54.507010 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.506910 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" event={"ID":"f7ebe0cc-858a-4af3-83c9-8a458547a335","Type":"ContainerDied","Data":"d40e8754d4b2eb68d5d47304e9bb8600c8b9c6d34ac55e2e507a16b86e66714b"} Sep 19 05:54:54.522110 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.522076 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4"] Sep 19 05:54:54.525556 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.525525 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:54.525975 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.525955 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0c7c7a36-jwdz4"] Sep 19 05:54:54.581651 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.581625 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:54.642171 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.642141 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" Sep 19 05:54:54.671447 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.671392 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:54.738364 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.738327 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7ebe0cc-858a-4af3-83c9-8a458547a335-plugins-config-volume\") pod \"f7ebe0cc-858a-4af3-83c9-8a458547a335\" (UID: \"f7ebe0cc-858a-4af3-83c9-8a458547a335\") " Sep 19 05:54:54.738828 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.738432 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb6lx\" (UniqueName: \"kubernetes.io/projected/f7ebe0cc-858a-4af3-83c9-8a458547a335-kube-api-access-fb6lx\") pod \"f7ebe0cc-858a-4af3-83c9-8a458547a335\" (UID: \"f7ebe0cc-858a-4af3-83c9-8a458547a335\") " Sep 19 05:54:54.738914 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.738819 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ebe0cc-858a-4af3-83c9-8a458547a335-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7ebe0cc-858a-4af3-83c9-8a458547a335" (UID: "f7ebe0cc-858a-4af3-83c9-8a458547a335"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:54:54.740699 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.740669 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7ebe0cc-858a-4af3-83c9-8a458547a335-kube-api-access-fb6lx" (OuterVolumeSpecName: "kube-api-access-fb6lx") pod "f7ebe0cc-858a-4af3-83c9-8a458547a335" (UID: "f7ebe0cc-858a-4af3-83c9-8a458547a335"). InnerVolumeSpecName "kube-api-access-fb6lx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:54:54.840298 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.840255 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7ebe0cc-858a-4af3-83c9-8a458547a335-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:54:54.840298 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.840290 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb6lx\" (UniqueName: \"kubernetes.io/projected/f7ebe0cc-858a-4af3-83c9-8a458547a335-kube-api-access-fb6lx\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:54:54.984836 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:54.984747 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:55.231526 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:55.231490 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:55.482143 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:55.482110 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:55.513548 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:55.513524 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" Sep 19 05:54:55.513718 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:55.513565 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd" event={"ID":"f7ebe0cc-858a-4af3-83c9-8a458547a335","Type":"ContainerDied","Data":"1ba6230aecd50c217459db4c31fac8afdd47ddc7762e5d3c4c3ff7ba31a12eb7"} Sep 19 05:54:55.513718 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:55.513628 2584 scope.go:117] "RemoveContainer" containerID="d40e8754d4b2eb68d5d47304e9bb8600c8b9c6d34ac55e2e507a16b86e66714b" Sep 19 05:54:55.535560 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:55.535530 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd"] Sep 19 05:54:55.539454 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:55.539430 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0c7c7a36-58fbdf4458-5r4vd"] Sep 19 05:54:55.887114 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:55.887080 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:55.983855 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:55.983819 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:56.267918 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:56.267830 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3f476bd-aa57-46c1-bf0c-95db65296b76" path="/var/lib/kubelet/pods/d3f476bd-aa57-46c1-bf0c-95db65296b76/volumes" Sep 19 05:54:56.268340 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:56.268319 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7ebe0cc-858a-4af3-83c9-8a458547a335" path="/var/lib/kubelet/pods/f7ebe0cc-858a-4af3-83c9-8a458547a335/volumes" Sep 19 05:54:56.784377 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:56.784344 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:57.796791 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:57.796766 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:58.536038 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:58.536001 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:58.638134 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:58.638104 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:54:59.582122 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:54:59.582088 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:00.528212 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:00.528173 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:00.629473 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:00.629439 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:00.726642 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:00.726602 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:01.202320 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.202287 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:01.336584 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.336547 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:01.477299 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.477221 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89"] Sep 19 05:55:01.477813 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.477792 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7ebe0cc-858a-4af3-83c9-8a458547a335" containerName="payload-processing" Sep 19 05:55:01.477813 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.477814 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7ebe0cc-858a-4af3-83c9-8a458547a335" containerName="payload-processing" Sep 19 05:55:01.477977 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.477827 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f476bd-aa57-46c1-bf0c-95db65296b76" containerName="revoke-keys" Sep 19 05:55:01.477977 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.477833 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f476bd-aa57-46c1-bf0c-95db65296b76" containerName="revoke-keys" Sep 19 05:55:01.477977 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.477937 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3f476bd-aa57-46c1-bf0c-95db65296b76" containerName="revoke-keys" Sep 19 05:55:01.477977 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.477953 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7ebe0cc-858a-4af3-83c9-8a458547a335" containerName="payload-processing" Sep 19 05:55:01.481055 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.481040 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" Sep 19 05:55:01.483896 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.483874 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-32a87fd8-dockercfg-7ppds\"" Sep 19 05:55:01.484159 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.484139 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-32a87fd8\"" Sep 19 05:55:01.492420 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.492329 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89"] Sep 19 05:55:01.609692 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.609650 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrczx\" (UniqueName: \"kubernetes.io/projected/8cb29d94-5329-4d74-994a-37bbb2443e08-kube-api-access-zrczx\") pod \"payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89\" (UID: \"8cb29d94-5329-4d74-994a-37bbb2443e08\") " pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" Sep 19 05:55:01.609888 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.609705 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cb29d94-5329-4d74-994a-37bbb2443e08-plugins-config-volume\") pod \"payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89\" (UID: \"8cb29d94-5329-4d74-994a-37bbb2443e08\") " pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" Sep 19 05:55:01.710904 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.710861 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrczx\" (UniqueName: \"kubernetes.io/projected/8cb29d94-5329-4d74-994a-37bbb2443e08-kube-api-access-zrczx\") pod \"payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89\" (UID: \"8cb29d94-5329-4d74-994a-37bbb2443e08\") " pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" Sep 19 05:55:01.711362 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.710928 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cb29d94-5329-4d74-994a-37bbb2443e08-plugins-config-volume\") pod \"payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89\" (UID: \"8cb29d94-5329-4d74-994a-37bbb2443e08\") " pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" Sep 19 05:55:01.711934 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.711907 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cb29d94-5329-4d74-994a-37bbb2443e08-plugins-config-volume\") pod \"payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89\" (UID: \"8cb29d94-5329-4d74-994a-37bbb2443e08\") " pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" Sep 19 05:55:01.719606 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.719543 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrczx\" (UniqueName: \"kubernetes.io/projected/8cb29d94-5329-4d74-994a-37bbb2443e08-kube-api-access-zrczx\") pod \"payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89\" (UID: \"8cb29d94-5329-4d74-994a-37bbb2443e08\") " pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" Sep 19 05:55:01.734170 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.734091 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:01.794753 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:01.794721 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" Sep 19 05:55:02.092837 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:02.092810 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89"] Sep 19 05:55:02.095506 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:55:02.095477 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb29d94_5329_4d74_994a_37bbb2443e08.slice/crio-0421cfba3e52fe784396b6f629f456dd059eac488186db9436e0230a29a356e8 WatchSource:0}: Error finding container 0421cfba3e52fe784396b6f629f456dd059eac488186db9436e0230a29a356e8: Status 404 returned error can't find the container with id 0421cfba3e52fe784396b6f629f456dd059eac488186db9436e0230a29a356e8 Sep 19 05:55:02.562721 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:02.562635 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" event={"ID":"8cb29d94-5329-4d74-994a-37bbb2443e08","Type":"ContainerStarted","Data":"9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d"} Sep 19 05:55:02.562721 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:02.562684 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" event={"ID":"8cb29d94-5329-4d74-994a-37bbb2443e08","Type":"ContainerStarted","Data":"0421cfba3e52fe784396b6f629f456dd059eac488186db9436e0230a29a356e8"} Sep 19 05:55:02.562721 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:02.562716 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" Sep 19 05:55:02.581915 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:02.581859 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" podStartSLOduration=1.5818433330000001 podStartE2EDuration="1.581843333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:55:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:55:02.579453216 +0000 UTC m=+830.572680748" watchObservedRunningTime="2026-09-19 05:55:02.581843333 +0000 UTC m=+830.575070863" Sep 19 05:55:02.626595 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:02.626560 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:03.405166 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:03.405136 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:04.250085 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:04.250050 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:05.108962 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:05.108921 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:09.384378 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:09.384347 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:13.572541 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:13.572506 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" Sep 19 05:55:25.457051 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:25.457014 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:26.968570 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:26.968541 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:27.797640 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:27.797603 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:28.569690 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:28.569659 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:29.343899 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:29.343842 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:29.454685 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:29.454649 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt"] Sep 19 05:55:29.458425 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:29.458384 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt" Sep 19 05:55:29.460643 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:29.460619 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 05:55:29.467031 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:29.467004 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt"] Sep 19 05:55:29.593810 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:29.593770 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11c2017c-7959-4eb2-9288-b1b169a8d9f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt\" (UID: \"11c2017c-7959-4eb2-9288-b1b169a8d9f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt" Sep 19 05:55:29.694805 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:29.694704 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11c2017c-7959-4eb2-9288-b1b169a8d9f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt\" (UID: \"11c2017c-7959-4eb2-9288-b1b169a8d9f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt" Sep 19 05:55:29.695331 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:29.695311 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11c2017c-7959-4eb2-9288-b1b169a8d9f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt\" (UID: \"11c2017c-7959-4eb2-9288-b1b169a8d9f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt" Sep 19 05:55:29.769950 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:29.769916 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt" Sep 19 05:55:30.043905 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:30.043874 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt"] Sep 19 05:55:30.045787 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:55:30.045758 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11c2017c_7959_4eb2_9288_b1b169a8d9f7.slice/crio-5558f4cd2c4319e648f179ff96fffa398ae71fbc208295820ac26ae911a7c3d3 WatchSource:0}: Error finding container 5558f4cd2c4319e648f179ff96fffa398ae71fbc208295820ac26ae911a7c3d3: Status 404 returned error can't find the container with id 5558f4cd2c4319e648f179ff96fffa398ae71fbc208295820ac26ae911a7c3d3 Sep 19 05:55:30.170045 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:30.170016 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:30.722722 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:30.722694 2584 generic.go:356] "Generic (PLEG): container finished" podID="11c2017c-7959-4eb2-9288-b1b169a8d9f7" containerID="bbb472763a13d3fac0acc3bf1b862da0572ab5d465727d5eb914ee99040357e6" exitCode=0 Sep 19 05:55:30.723025 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:30.722753 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt" event={"ID":"11c2017c-7959-4eb2-9288-b1b169a8d9f7","Type":"ContainerDied","Data":"bbb472763a13d3fac0acc3bf1b862da0572ab5d465727d5eb914ee99040357e6"} Sep 19 05:55:30.723025 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:30.722793 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt" event={"ID":"11c2017c-7959-4eb2-9288-b1b169a8d9f7","Type":"ContainerStarted","Data":"5558f4cd2c4319e648f179ff96fffa398ae71fbc208295820ac26ae911a7c3d3"} Sep 19 05:55:30.993040 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:30.992959 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:31.709973 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:31.709935 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:31.943271 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:31.943244 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt" Sep 19 05:55:32.019784 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:32.019702 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11c2017c-7959-4eb2-9288-b1b169a8d9f7-maas-api-service-ca\") pod \"11c2017c-7959-4eb2-9288-b1b169a8d9f7\" (UID: \"11c2017c-7959-4eb2-9288-b1b169a8d9f7\") " Sep 19 05:55:32.020128 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:32.020100 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11c2017c-7959-4eb2-9288-b1b169a8d9f7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "11c2017c-7959-4eb2-9288-b1b169a8d9f7" (UID: "11c2017c-7959-4eb2-9288-b1b169a8d9f7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:55:32.120746 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:32.120699 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11c2017c-7959-4eb2-9288-b1b169a8d9f7-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:55:32.517391 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:32.517360 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:32.738134 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:32.738108 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt" Sep 19 05:55:32.738134 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:32.738122 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt" event={"ID":"11c2017c-7959-4eb2-9288-b1b169a8d9f7","Type":"ContainerDied","Data":"5558f4cd2c4319e648f179ff96fffa398ae71fbc208295820ac26ae911a7c3d3"} Sep 19 05:55:32.738337 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:32.738168 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5558f4cd2c4319e648f179ff96fffa398ae71fbc208295820ac26ae911a7c3d3" Sep 19 05:55:33.226784 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:33.226753 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:34.079149 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:34.079117 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:34.771345 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:34.771309 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:40.031487 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:40.031447 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:40.798384 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:40.798353 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:40.822618 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:40.822581 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89"] Sep 19 05:55:40.822886 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:40.822857 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" podUID="8cb29d94-5329-4d74-994a-37bbb2443e08" containerName="payload-processing" containerID="cri-o://9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d" gracePeriod=30 Sep 19 05:55:40.870752 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:40.870724 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:40.968959 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:40.968921 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:40.997540 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:40.997435 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt"] Sep 19 05:55:41.004057 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.004027 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-32a87fd8-hn8dt"] Sep 19 05:55:41.034238 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.034206 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:41.104736 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.104697 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" Sep 19 05:55:41.211810 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.211780 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrczx\" (UniqueName: \"kubernetes.io/projected/8cb29d94-5329-4d74-994a-37bbb2443e08-kube-api-access-zrczx\") pod \"8cb29d94-5329-4d74-994a-37bbb2443e08\" (UID: \"8cb29d94-5329-4d74-994a-37bbb2443e08\") " Sep 19 05:55:41.212016 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.211907 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cb29d94-5329-4d74-994a-37bbb2443e08-plugins-config-volume\") pod \"8cb29d94-5329-4d74-994a-37bbb2443e08\" (UID: \"8cb29d94-5329-4d74-994a-37bbb2443e08\") " Sep 19 05:55:41.212316 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.212291 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cb29d94-5329-4d74-994a-37bbb2443e08-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8cb29d94-5329-4d74-994a-37bbb2443e08" (UID: "8cb29d94-5329-4d74-994a-37bbb2443e08"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:55:41.214052 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.214029 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cb29d94-5329-4d74-994a-37bbb2443e08-kube-api-access-zrczx" (OuterVolumeSpecName: "kube-api-access-zrczx") pod "8cb29d94-5329-4d74-994a-37bbb2443e08" (UID: "8cb29d94-5329-4d74-994a-37bbb2443e08"). InnerVolumeSpecName "kube-api-access-zrczx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:55:41.313956 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.313905 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cb29d94-5329-4d74-994a-37bbb2443e08-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:55:41.313956 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.313948 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrczx\" (UniqueName: \"kubernetes.io/projected/8cb29d94-5329-4d74-994a-37bbb2443e08-kube-api-access-zrczx\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:55:41.380044 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.380012 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:41.790757 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.790673 2584 generic.go:356] "Generic (PLEG): container finished" podID="8cb29d94-5329-4d74-994a-37bbb2443e08" containerID="9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d" exitCode=0 Sep 19 05:55:41.790757 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.790720 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" Sep 19 05:55:41.790942 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.790755 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" event={"ID":"8cb29d94-5329-4d74-994a-37bbb2443e08","Type":"ContainerDied","Data":"9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d"} Sep 19 05:55:41.790942 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.790807 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89" event={"ID":"8cb29d94-5329-4d74-994a-37bbb2443e08","Type":"ContainerDied","Data":"0421cfba3e52fe784396b6f629f456dd059eac488186db9436e0230a29a356e8"} Sep 19 05:55:41.790942 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.790831 2584 scope.go:117] "RemoveContainer" containerID="9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d" Sep 19 05:55:41.801113 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.801093 2584 scope.go:117] "RemoveContainer" containerID="9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d" Sep 19 05:55:41.801433 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:55:41.801409 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d\": container with ID starting with 9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d not found: ID does not exist" containerID="9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d" Sep 19 05:55:41.801513 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.801443 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d"} err="failed to get container status \"9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d\": rpc error: code = NotFound desc = could not find container \"9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d\": container with ID starting with 9d3fd61548c756588268396a1d30b9a7f2587db41bf74148376915a14b23de7d not found: ID does not exist" Sep 19 05:55:41.812007 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.811980 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89"] Sep 19 05:55:41.818005 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:41.817984 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-32a87fd8-698458f6d5-m9r89"] Sep 19 05:55:42.181158 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.181126 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:42.267293 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.267257 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11c2017c-7959-4eb2-9288-b1b169a8d9f7" path="/var/lib/kubelet/pods/11c2017c-7959-4eb2-9288-b1b169a8d9f7/volumes" Sep 19 05:55:42.267695 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.267680 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cb29d94-5329-4d74-994a-37bbb2443e08" path="/var/lib/kubelet/pods/8cb29d94-5329-4d74-994a-37bbb2443e08/volumes" Sep 19 05:55:42.281458 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.281427 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:42.319443 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.319411 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx"] Sep 19 05:55:42.319891 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.319879 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11c2017c-7959-4eb2-9288-b1b169a8d9f7" containerName="revoke-keys" Sep 19 05:55:42.319943 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.319892 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="11c2017c-7959-4eb2-9288-b1b169a8d9f7" containerName="revoke-keys" Sep 19 05:55:42.319943 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.319907 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb29d94-5329-4d74-994a-37bbb2443e08" containerName="payload-processing" Sep 19 05:55:42.319943 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.319912 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb29d94-5329-4d74-994a-37bbb2443e08" containerName="payload-processing" Sep 19 05:55:42.320040 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.319985 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cb29d94-5329-4d74-994a-37bbb2443e08" containerName="payload-processing" Sep 19 05:55:42.320040 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.319998 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="11c2017c-7959-4eb2-9288-b1b169a8d9f7" containerName="revoke-keys" Sep 19 05:55:42.324626 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.324603 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx" Sep 19 05:55:42.326851 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.326830 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 05:55:42.331047 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.331022 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx"] Sep 19 05:55:42.426664 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.426633 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69aa2c8f-dc69-426f-a248-847ebb86d252-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-8128f3-w4lxx\" (UID: \"69aa2c8f-dc69-426f-a248-847ebb86d252\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx" Sep 19 05:55:42.527762 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.527671 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69aa2c8f-dc69-426f-a248-847ebb86d252-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-8128f3-w4lxx\" (UID: \"69aa2c8f-dc69-426f-a248-847ebb86d252\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx" Sep 19 05:55:42.528372 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.528351 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69aa2c8f-dc69-426f-a248-847ebb86d252-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-8128f3-w4lxx\" (UID: \"69aa2c8f-dc69-426f-a248-847ebb86d252\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx" Sep 19 05:55:42.638864 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.638829 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx" Sep 19 05:55:42.679307 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.679273 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:42.921687 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.921663 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx"] Sep 19 05:55:42.923738 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:55:42.923707 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69aa2c8f_dc69_426f_a248_847ebb86d252.slice/crio-a800d1807aa422a22b80898766c62d2b4903e3b4f1852040c397630c2395db6d WatchSource:0}: Error finding container a800d1807aa422a22b80898766c62d2b4903e3b4f1852040c397630c2395db6d: Status 404 returned error can't find the container with id a800d1807aa422a22b80898766c62d2b4903e3b4f1852040c397630c2395db6d Sep 19 05:55:42.977831 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.977795 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw"] Sep 19 05:55:42.982808 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.982783 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:42.985272 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.985248 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-39e34e62-openshift-default-dockercfg-vzfdd\"" Sep 19 05:55:42.995050 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:42.995024 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw"] Sep 19 05:55:43.004884 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.004861 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:43.032502 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.032471 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istiod-ca-cert\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.032654 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.032503 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-workload-socket\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.032654 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.032551 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-podinfo\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.032654 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.032614 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-envoy\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.032654 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.032639 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-data\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.032874 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.032657 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-token\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.032874 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.032801 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-workload-certs\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.032982 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.032878 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8njnq\" (UniqueName: \"kubernetes.io/projected/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-kube-api-access-8njnq\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.032982 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.032919 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-credential-socket\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.123653 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.123579 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:43.133965 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.133940 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-workload-certs\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134072 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.133991 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8njnq\" (UniqueName: \"kubernetes.io/projected/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-kube-api-access-8njnq\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134072 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.134018 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-credential-socket\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134072 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.134048 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istiod-ca-cert\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134072 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.134065 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-workload-socket\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134295 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.134092 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-podinfo\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134295 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.134203 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-envoy\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134295 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.134228 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-data\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134295 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.134249 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-token\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134555 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.134307 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-workload-certs\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134617 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.134547 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-workload-socket\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134697 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.134677 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-data\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134814 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.134796 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istiod-ca-cert\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.134926 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.134906 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-credential-socket\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.136616 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.136591 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-envoy\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.136963 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.136946 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-podinfo\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.142661 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.142639 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-token\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.142916 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.142900 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8njnq\" (UniqueName: \"kubernetes.io/projected/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-kube-api-access-8njnq\") pod \"e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.294215 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.294177 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:43.594052 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.593970 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw"] Sep 19 05:55:43.596242 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:55:43.596219 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4fb9a55_ffa2_41f7_9f33_6a1a9c9bbeb7.slice/crio-5b3e3a233918920dd03bb290990c2dff5d8db8da30f916c7f383f8c61954b321 WatchSource:0}: Error finding container 5b3e3a233918920dd03bb290990c2dff5d8db8da30f916c7f383f8c61954b321: Status 404 returned error can't find the container with id 5b3e3a233918920dd03bb290990c2dff5d8db8da30f916c7f383f8c61954b321 Sep 19 05:55:43.598864 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.598823 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:55:43.598948 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.598899 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:55:43.598948 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.598932 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:55:43.621185 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.621161 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:43.805993 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.805942 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" event={"ID":"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7","Type":"ContainerStarted","Data":"6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a"} Sep 19 05:55:43.806172 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.806033 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" event={"ID":"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7","Type":"ContainerStarted","Data":"5b3e3a233918920dd03bb290990c2dff5d8db8da30f916c7f383f8c61954b321"} Sep 19 05:55:43.807916 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.807890 2584 generic.go:356] "Generic (PLEG): container finished" podID="69aa2c8f-dc69-426f-a248-847ebb86d252" containerID="bdfb09263db38b08b68c67da69509bb42606cafee6c17d72ffe3a135a7c1a583" exitCode=0 Sep 19 05:55:43.808062 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.807976 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx" event={"ID":"69aa2c8f-dc69-426f-a248-847ebb86d252","Type":"ContainerDied","Data":"bdfb09263db38b08b68c67da69509bb42606cafee6c17d72ffe3a135a7c1a583"} Sep 19 05:55:43.808062 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.808027 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx" event={"ID":"69aa2c8f-dc69-426f-a248-847ebb86d252","Type":"ContainerStarted","Data":"a800d1807aa422a22b80898766c62d2b4903e3b4f1852040c397630c2395db6d"} Sep 19 05:55:43.830739 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:43.830690 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" podStartSLOduration=1.830675147 podStartE2EDuration="1.830675147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:55:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:55:43.827828332 +0000 UTC m=+871.821055864" watchObservedRunningTime="2026-09-19 05:55:43.830675147 +0000 UTC m=+871.823902750" Sep 19 05:55:44.119642 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:44.119611 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:44.294705 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:44.294674 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:44.299984 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:44.299955 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:44.814143 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:44.814107 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:44.815113 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:44.815084 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:55:44.918621 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:44.918591 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:44.966960 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:44.966934 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx" Sep 19 05:55:45.055275 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:45.055235 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69aa2c8f-dc69-426f-a248-847ebb86d252-maas-api-service-ca\") pod \"69aa2c8f-dc69-426f-a248-847ebb86d252\" (UID: \"69aa2c8f-dc69-426f-a248-847ebb86d252\") " Sep 19 05:55:45.055653 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:45.055626 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69aa2c8f-dc69-426f-a248-847ebb86d252-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "69aa2c8f-dc69-426f-a248-847ebb86d252" (UID: "69aa2c8f-dc69-426f-a248-847ebb86d252"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:55:45.055731 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:45.055716 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69aa2c8f-dc69-426f-a248-847ebb86d252-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:55:45.253852 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:45.253821 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:45.355839 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:45.355806 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:45.808070 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:45.808044 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:45.820900 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:45.820869 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx" event={"ID":"69aa2c8f-dc69-426f-a248-847ebb86d252","Type":"ContainerDied","Data":"a800d1807aa422a22b80898766c62d2b4903e3b4f1852040c397630c2395db6d"} Sep 19 05:55:45.820900 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:45.820892 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx" Sep 19 05:55:45.821080 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:45.820941 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a800d1807aa422a22b80898766c62d2b4903e3b4f1852040c397630c2395db6d" Sep 19 05:55:46.927560 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:46.927533 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:47.263358 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:47.263265 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:47.638506 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:47.638469 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:48.511792 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:48.511764 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:48.610552 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:48.610523 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:48.710836 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:48.710804 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:48.973081 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:48.973048 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:49.067779 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:49.067750 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:49.394600 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:49.394563 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:50.094750 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:50.094720 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:50.805313 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:50.805283 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:51.660281 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:51.660250 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:52.512635 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.512601 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:52.545580 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.545543 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f"] Sep 19 05:55:52.545858 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.545829 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" podUID="03ab01be-ae18-4ef2-91e0-b8c21222d49e" containerName="payload-processing" containerID="cri-o://c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c" gracePeriod=30 Sep 19 05:55:52.609452 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.609422 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:52.708806 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.708770 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:52.774509 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.774409 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx"] Sep 19 05:55:52.777528 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.777498 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8128f3-w4lxx"] Sep 19 05:55:52.800499 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.800471 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:52.820370 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.820341 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" Sep 19 05:55:52.864142 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.864113 2584 generic.go:356] "Generic (PLEG): container finished" podID="03ab01be-ae18-4ef2-91e0-b8c21222d49e" containerID="c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c" exitCode=0 Sep 19 05:55:52.864310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.864179 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" Sep 19 05:55:52.864310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.864203 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" event={"ID":"03ab01be-ae18-4ef2-91e0-b8c21222d49e","Type":"ContainerDied","Data":"c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c"} Sep 19 05:55:52.864310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.864265 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f" event={"ID":"03ab01be-ae18-4ef2-91e0-b8c21222d49e","Type":"ContainerDied","Data":"2fcfe20b1b9d98d4ceb23fa4f8208705264a5d2e402175daff87ded54763c0f6"} Sep 19 05:55:52.864310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.864294 2584 scope.go:117] "RemoveContainer" containerID="c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c" Sep 19 05:55:52.874444 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.874382 2584 scope.go:117] "RemoveContainer" containerID="c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c" Sep 19 05:55:52.874800 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:55:52.874773 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c\": container with ID starting with c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c not found: ID does not exist" containerID="c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c" Sep 19 05:55:52.874913 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.874812 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c"} err="failed to get container status \"c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c\": rpc error: code = NotFound desc = could not find container \"c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c\": container with ID starting with c4f150945f0b07f4b942be18a2035b73f655a963a65b420b96b211d093772c1c not found: ID does not exist" Sep 19 05:55:52.941529 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.941489 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpzkd\" (UniqueName: \"kubernetes.io/projected/03ab01be-ae18-4ef2-91e0-b8c21222d49e-kube-api-access-kpzkd\") pod \"03ab01be-ae18-4ef2-91e0-b8c21222d49e\" (UID: \"03ab01be-ae18-4ef2-91e0-b8c21222d49e\") " Sep 19 05:55:52.941731 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.941569 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03ab01be-ae18-4ef2-91e0-b8c21222d49e-plugins-config-volume\") pod \"03ab01be-ae18-4ef2-91e0-b8c21222d49e\" (UID: \"03ab01be-ae18-4ef2-91e0-b8c21222d49e\") " Sep 19 05:55:52.941927 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.941896 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03ab01be-ae18-4ef2-91e0-b8c21222d49e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "03ab01be-ae18-4ef2-91e0-b8c21222d49e" (UID: "03ab01be-ae18-4ef2-91e0-b8c21222d49e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:55:52.943675 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:52.943651 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03ab01be-ae18-4ef2-91e0-b8c21222d49e-kube-api-access-kpzkd" (OuterVolumeSpecName: "kube-api-access-kpzkd") pod "03ab01be-ae18-4ef2-91e0-b8c21222d49e" (UID: "03ab01be-ae18-4ef2-91e0-b8c21222d49e"). InnerVolumeSpecName "kube-api-access-kpzkd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:55:53.042978 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:53.042895 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpzkd\" (UniqueName: \"kubernetes.io/projected/03ab01be-ae18-4ef2-91e0-b8c21222d49e-kube-api-access-kpzkd\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:55:53.042978 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:53.042923 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03ab01be-ae18-4ef2-91e0-b8c21222d49e-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:55:53.152686 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:53.152647 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:53.185575 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:53.185544 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f"] Sep 19 05:55:53.189631 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:53.189604 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8128f3-7b47db56fc-5425f"] Sep 19 05:55:53.408713 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:53.408682 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:54.267834 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:54.267805 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03ab01be-ae18-4ef2-91e0-b8c21222d49e" path="/var/lib/kubelet/pods/03ab01be-ae18-4ef2-91e0-b8c21222d49e/volumes" Sep 19 05:55:54.268262 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:54.268130 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69aa2c8f-dc69-426f-a248-847ebb86d252" path="/var/lib/kubelet/pods/69aa2c8f-dc69-426f-a248-847ebb86d252/volumes" Sep 19 05:55:54.308326 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:54.308290 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:54.999089 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:54.999044 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:55.687628 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:55.687600 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:56.467858 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:56.467815 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:57.236530 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:57.236484 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:58.231362 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:58.231324 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:58.696632 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:58.696595 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:59.396920 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:59.396885 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:55:59.825008 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:59.824926 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn"] Sep 19 05:55:59.825477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:59.825459 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69aa2c8f-dc69-426f-a248-847ebb86d252" containerName="revoke-keys" Sep 19 05:55:59.825477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:59.825473 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="69aa2c8f-dc69-426f-a248-847ebb86d252" containerName="revoke-keys" Sep 19 05:55:59.825566 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:59.825503 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03ab01be-ae18-4ef2-91e0-b8c21222d49e" containerName="payload-processing" Sep 19 05:55:59.825566 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:59.825511 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="03ab01be-ae18-4ef2-91e0-b8c21222d49e" containerName="payload-processing" Sep 19 05:55:59.825629 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:59.825600 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="69aa2c8f-dc69-426f-a248-847ebb86d252" containerName="revoke-keys" Sep 19 05:55:59.825629 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:59.825613 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="03ab01be-ae18-4ef2-91e0-b8c21222d49e" containerName="payload-processing" Sep 19 05:55:59.828810 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:59.828788 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn" Sep 19 05:55:59.831119 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:59.831098 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 05:55:59.836042 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:59.836007 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn"] Sep 19 05:55:59.910163 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:55:59.910116 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c32ffdf-4883-4cab-82cf-18829c598b9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-39e34e62-66kwn\" (UID: \"8c32ffdf-4883-4cab-82cf-18829c598b9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn" Sep 19 05:56:00.011583 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:00.011535 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c32ffdf-4883-4cab-82cf-18829c598b9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-39e34e62-66kwn\" (UID: \"8c32ffdf-4883-4cab-82cf-18829c598b9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn" Sep 19 05:56:00.012447 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:00.012418 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c32ffdf-4883-4cab-82cf-18829c598b9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-39e34e62-66kwn\" (UID: \"8c32ffdf-4883-4cab-82cf-18829c598b9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn" Sep 19 05:56:00.142423 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:00.142352 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn" Sep 19 05:56:00.421258 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:00.421219 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn"] Sep 19 05:56:00.423992 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:56:00.423954 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c32ffdf_4883_4cab_82cf_18829c598b9c.slice/crio-ca06f8f1a243bf720c78080ac9bc207d845ce3e588e2154680a97e31a87d2cd6 WatchSource:0}: Error finding container ca06f8f1a243bf720c78080ac9bc207d845ce3e588e2154680a97e31a87d2cd6: Status 404 returned error can't find the container with id ca06f8f1a243bf720c78080ac9bc207d845ce3e588e2154680a97e31a87d2cd6 Sep 19 05:56:00.579755 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:00.579718 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:00.913491 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:00.913452 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn" event={"ID":"8c32ffdf-4883-4cab-82cf-18829c598b9c","Type":"ContainerStarted","Data":"f82cf4c0e97719114316f22056c6c6f6282d8fa30dd4f0eb2343d837e12e8f7e"} Sep 19 05:56:00.913883 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:00.913521 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn" event={"ID":"8c32ffdf-4883-4cab-82cf-18829c598b9c","Type":"ContainerStarted","Data":"ca06f8f1a243bf720c78080ac9bc207d845ce3e588e2154680a97e31a87d2cd6"} Sep 19 05:56:00.928674 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:00.928611 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn" podStartSLOduration=1.928550854 podStartE2EDuration="1.928550854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:56:00.927830062 +0000 UTC m=+888.921057593" watchObservedRunningTime="2026-09-19 05:56:00.928550854 +0000 UTC m=+888.921778389" Sep 19 05:56:01.449044 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:01.448982 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:01.925071 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:01.925037 2584 generic.go:356] "Generic (PLEG): container finished" podID="8c32ffdf-4883-4cab-82cf-18829c598b9c" containerID="f82cf4c0e97719114316f22056c6c6f6282d8fa30dd4f0eb2343d837e12e8f7e" exitCode=0 Sep 19 05:56:01.925731 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:01.925218 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn" event={"ID":"8c32ffdf-4883-4cab-82cf-18829c598b9c","Type":"ContainerDied","Data":"f82cf4c0e97719114316f22056c6c6f6282d8fa30dd4f0eb2343d837e12e8f7e"} Sep 19 05:56:02.233182 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:02.233086 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:03.096214 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:03.096187 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn" Sep 19 05:56:03.231326 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:03.231021 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:03.250706 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:03.250672 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c32ffdf-4883-4cab-82cf-18829c598b9c-maas-api-service-ca\") pod \"8c32ffdf-4883-4cab-82cf-18829c598b9c\" (UID: \"8c32ffdf-4883-4cab-82cf-18829c598b9c\") " Sep 19 05:56:03.251032 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:03.251004 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c32ffdf-4883-4cab-82cf-18829c598b9c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8c32ffdf-4883-4cab-82cf-18829c598b9c" (UID: "8c32ffdf-4883-4cab-82cf-18829c598b9c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:56:03.251147 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:03.251133 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c32ffdf-4883-4cab-82cf-18829c598b9c-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:56:03.943666 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:03.943627 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn" event={"ID":"8c32ffdf-4883-4cab-82cf-18829c598b9c","Type":"ContainerDied","Data":"ca06f8f1a243bf720c78080ac9bc207d845ce3e588e2154680a97e31a87d2cd6"} Sep 19 05:56:03.943858 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:03.943679 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca06f8f1a243bf720c78080ac9bc207d845ce3e588e2154680a97e31a87d2cd6" Sep 19 05:56:03.943858 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:03.943644 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn" Sep 19 05:56:04.044588 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:04.044525 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:04.588463 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:04.588429 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:04.835019 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:04.834987 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:04.933477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:04.933368 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:05.319816 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:05.319711 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:05.774914 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:05.774877 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:06.181448 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:06.181385 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:06.891253 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:06.891210 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:08.015187 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:08.015137 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:08.681090 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:08.681061 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:08.761741 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:08.761690 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:09.027917 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:09.027819 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:09.801244 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:09.801155 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:09.979185 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:09.979154 2584 generic.go:356] "Generic (PLEG): container finished" podID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerID="6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a" exitCode=0 Sep 19 05:56:09.979349 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:09.979235 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" event={"ID":"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7","Type":"ContainerDied","Data":"6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a"} Sep 19 05:56:09.979713 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:09.979699 2584 scope.go:117] "RemoveContainer" containerID="6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a" Sep 19 05:56:09.980895 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:09.980864 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:56:09.981012 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:09.980934 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:56:09.981012 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:09.980967 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:56:10.810241 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:10.810197 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:10.986845 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:10.986811 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" event={"ID":"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7","Type":"ContainerStarted","Data":"b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2"} Sep 19 05:56:11.295009 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:11.294975 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:56:11.296033 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:11.296006 2584 patch_prober.go:28] interesting pod/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 19 05:56:11.296100 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:11.296077 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" podUID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 19 05:56:11.604092 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:11.604036 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:12.294832 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:12.294796 2584 patch_prober.go:28] interesting pod/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 19 05:56:12.295226 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:12.294861 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" podUID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 19 05:56:12.601622 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:12.601590 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:13.295338 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:13.295303 2584 patch_prober.go:28] interesting pod/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 19 05:56:13.295850 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:13.295363 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" podUID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 19 05:56:13.439032 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:13.438996 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:13.543999 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:13.543958 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:13.656657 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:13.656620 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:13.990246 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:13.990147 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:14.089057 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.089022 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:14.120423 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.106988 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt"] Sep 19 05:56:14.129224 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.129190 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c32ffdf-4883-4cab-82cf-18829c598b9c" containerName="revoke-keys" Sep 19 05:56:14.129462 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.129446 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c32ffdf-4883-4cab-82cf-18829c598b9c" containerName="revoke-keys" Sep 19 05:56:14.129749 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.129730 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c32ffdf-4883-4cab-82cf-18829c598b9c" containerName="revoke-keys" Sep 19 05:56:14.137431 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.137232 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt"] Sep 19 05:56:14.137431 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.137352 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" Sep 19 05:56:14.139727 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.139690 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-875220\"" Sep 19 05:56:14.139850 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.139768 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-875220-dockercfg-6xngb\"" Sep 19 05:56:14.247078 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.247002 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 05:56:14.247911 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.247891 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 05:56:14.271093 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.271054 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c052a17-232c-4bd6-8ef9-e94a8d12bc32-plugins-config-volume\") pod \"payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt\" (UID: \"1c052a17-232c-4bd6-8ef9-e94a8d12bc32\") " pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" Sep 19 05:56:14.271297 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.271201 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcxpf\" (UniqueName: \"kubernetes.io/projected/1c052a17-232c-4bd6-8ef9-e94a8d12bc32-kube-api-access-lcxpf\") pod \"payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt\" (UID: \"1c052a17-232c-4bd6-8ef9-e94a8d12bc32\") " pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" Sep 19 05:56:14.295156 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.295126 2584 patch_prober.go:28] interesting pod/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 19 05:56:14.295305 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.295185 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" podUID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 19 05:56:14.338929 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.338855 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:14.373152 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.372847 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c052a17-232c-4bd6-8ef9-e94a8d12bc32-plugins-config-volume\") pod \"payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt\" (UID: \"1c052a17-232c-4bd6-8ef9-e94a8d12bc32\") " pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" Sep 19 05:56:14.373152 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.372947 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcxpf\" (UniqueName: \"kubernetes.io/projected/1c052a17-232c-4bd6-8ef9-e94a8d12bc32-kube-api-access-lcxpf\") pod \"payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt\" (UID: \"1c052a17-232c-4bd6-8ef9-e94a8d12bc32\") " pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" Sep 19 05:56:14.373877 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.373848 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c052a17-232c-4bd6-8ef9-e94a8d12bc32-plugins-config-volume\") pod \"payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt\" (UID: \"1c052a17-232c-4bd6-8ef9-e94a8d12bc32\") " pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" Sep 19 05:56:14.389601 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.389573 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcxpf\" (UniqueName: \"kubernetes.io/projected/1c052a17-232c-4bd6-8ef9-e94a8d12bc32-kube-api-access-lcxpf\") pod \"payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt\" (UID: \"1c052a17-232c-4bd6-8ef9-e94a8d12bc32\") " pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" Sep 19 05:56:14.417178 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.417095 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:14.454170 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.454143 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" Sep 19 05:56:14.514479 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.512912 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:14.571828 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.571780 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn"] Sep 19 05:56:14.577782 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.577751 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-39e34e62-66kwn"] Sep 19 05:56:14.580234 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.580209 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:14.705589 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.705430 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw"] Sep 19 05:56:14.705789 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.705664 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" podUID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerName="istio-proxy" containerID="cri-o://b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2" gracePeriod=30 Sep 19 05:56:14.753864 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.753719 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:14.774306 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.774283 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt"] Sep 19 05:56:14.776560 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:56:14.776520 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c052a17_232c_4bd6_8ef9_e94a8d12bc32.slice/crio-b3eb64dd8bcd7cb7aab77e5bafdb7083b1c17352685d37cca57f4e0f0220736e WatchSource:0}: Error finding container b3eb64dd8bcd7cb7aab77e5bafdb7083b1c17352685d37cca57f4e0f0220736e: Status 404 returned error can't find the container with id b3eb64dd8bcd7cb7aab77e5bafdb7083b1c17352685d37cca57f4e0f0220736e Sep 19 05:56:14.821054 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:14.820744 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:56:15.015180 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:15.015118 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" event={"ID":"1c052a17-232c-4bd6-8ef9-e94a8d12bc32","Type":"ContainerStarted","Data":"935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e"} Sep 19 05:56:15.015389 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:15.015189 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" event={"ID":"1c052a17-232c-4bd6-8ef9-e94a8d12bc32","Type":"ContainerStarted","Data":"b3eb64dd8bcd7cb7aab77e5bafdb7083b1c17352685d37cca57f4e0f0220736e"} Sep 19 05:56:15.015638 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:15.015576 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" Sep 19 05:56:15.429574 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:15.429531 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:15.526585 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:15.526490 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:15.631136 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:15.631094 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:16.267699 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:16.267668 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c32ffdf-4883-4cab-82cf-18829c598b9c" path="/var/lib/kubelet/pods/8c32ffdf-4883-4cab-82cf-18829c598b9c/volumes" Sep 19 05:56:16.618606 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:16.618566 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:17.135892 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:17.135852 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:17.490858 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:17.490757 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:18.187779 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:18.187743 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:18.844669 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:18.844642 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:19.693732 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:19.693700 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:22.357540 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.357513 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:56:22.379779 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.379724 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" podStartSLOduration=8.379707095 podStartE2EDuration="8.379707095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:56:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:56:15.036994651 +0000 UTC m=+903.030222182" watchObservedRunningTime="2026-09-19 05:56:22.379707095 +0000 UTC m=+910.372934622" Sep 19 05:56:22.450938 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.450895 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-data\") pod \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " Sep 19 05:56:22.450938 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.450941 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8njnq\" (UniqueName: \"kubernetes.io/projected/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-kube-api-access-8njnq\") pod \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " Sep 19 05:56:22.451190 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.450962 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-podinfo\") pod \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " Sep 19 05:56:22.451190 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.450995 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-workload-certs\") pod \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " Sep 19 05:56:22.451190 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451086 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-token\") pod \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " Sep 19 05:56:22.451190 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451145 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-credential-socket\") pod \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " Sep 19 05:56:22.451426 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451231 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istiod-ca-cert\") pod \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " Sep 19 05:56:22.451426 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451268 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-data" (OuterVolumeSpecName: "istio-data") pod "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" (UID: "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 05:56:22.451426 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451299 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" (UID: "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 05:56:22.451426 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451281 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-envoy\") pod \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " Sep 19 05:56:22.451633 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451425 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-workload-socket\") pod \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\" (UID: \"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7\") " Sep 19 05:56:22.451633 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451538 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" (UID: "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 05:56:22.451738 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451637 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" (UID: "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:56:22.451784 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451742 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" (UID: "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 05:56:22.451945 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451926 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-credential-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:56:22.452026 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451952 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istiod-ca-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:56:22.452026 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451968 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-workload-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:56:22.452026 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451981 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-data\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:56:22.452026 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.451994 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-workload-certs\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:56:22.453295 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.453269 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-kube-api-access-8njnq" (OuterVolumeSpecName: "kube-api-access-8njnq") pod "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" (UID: "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7"). InnerVolumeSpecName "kube-api-access-8njnq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:56:22.453462 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.453440 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" (UID: "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 05:56:22.453765 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.453742 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" (UID: "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 05:56:22.453865 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.453765 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" (UID: "a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:56:22.553242 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.553155 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-envoy\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:56:22.553242 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.553185 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8njnq\" (UniqueName: \"kubernetes.io/projected/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-kube-api-access-8njnq\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:56:22.553242 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.553196 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-podinfo\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:56:22.553242 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.553205 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7-istio-token\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:56:22.714774 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.714744 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:22.930066 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:22.930031 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:23.033818 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.033786 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:23.068922 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.068892 2584 generic.go:356] "Generic (PLEG): container finished" podID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerID="b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2" exitCode=0 Sep 19 05:56:23.069096 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.068952 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" Sep 19 05:56:23.069096 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.068969 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" event={"ID":"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7","Type":"ContainerDied","Data":"b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2"} Sep 19 05:56:23.069096 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.069022 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw" event={"ID":"a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7","Type":"ContainerDied","Data":"5b3e3a233918920dd03bb290990c2dff5d8db8da30f916c7f383f8c61954b321"} Sep 19 05:56:23.069096 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.069045 2584 scope.go:117] "RemoveContainer" containerID="b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2" Sep 19 05:56:23.080159 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.080137 2584 scope.go:117] "RemoveContainer" containerID="6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a" Sep 19 05:56:23.088851 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.088835 2584 scope.go:117] "RemoveContainer" containerID="b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2" Sep 19 05:56:23.089130 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:56:23.089113 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2\": container with ID starting with b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2 not found: ID does not exist" containerID="b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2" Sep 19 05:56:23.089192 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.089139 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2"} err="failed to get container status \"b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2\": rpc error: code = NotFound desc = could not find container \"b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2\": container with ID starting with b5ddf160842690dd8d0666cfd298816b3e5abe41a439fb06472e79fc1579d7c2 not found: ID does not exist" Sep 19 05:56:23.089192 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.089157 2584 scope.go:117] "RemoveContainer" containerID="6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a" Sep 19 05:56:23.089457 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:56:23.089433 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a\": container with ID starting with 6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a not found: ID does not exist" containerID="6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a" Sep 19 05:56:23.089506 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.089464 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a"} err="failed to get container status \"6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a\": rpc error: code = NotFound desc = could not find container \"6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a\": container with ID starting with 6c0fb30aa9a02cd833f0e56e7e7edd48de77b2640ff30f828180640c735f723a not found: ID does not exist" Sep 19 05:56:23.095732 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.095709 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw"] Sep 19 05:56:23.100253 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.100233 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-39e34e62-openshift-default-7b6d986896-6sxkw"] Sep 19 05:56:23.134538 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.134511 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:23.234008 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.233921 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:23.759241 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:23.759196 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:24.267645 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:24.267612 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" path="/var/lib/kubelet/pods/a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7/volumes" Sep 19 05:56:26.027753 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:26.027723 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" Sep 19 05:56:31.915758 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:31.915724 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:32.693142 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:32.693111 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:36.638622 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:36.638587 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:37.412871 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:37.412839 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:38.102711 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:38.102681 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:38.775530 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:38.775494 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:38.875152 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:38.875122 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:39.809607 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:39.809570 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:40.572222 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:40.572193 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:47.651665 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:47.651631 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:49.621659 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:49.621626 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:55.497747 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:55.497711 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:55.610477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:55.610440 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:56:55.734178 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:56:55.734148 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:01.176063 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:01.176032 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:11.625975 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:11.625938 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:17.364591 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:17.364549 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:17.465792 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:17.465754 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:17.562392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:17.562357 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:25.460096 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:25.460017 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:28.115062 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:28.115021 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:29.366466 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:29.366434 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:35.402780 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:35.402742 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:35.504282 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:35.504246 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:35.604172 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:35.604141 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:35.670677 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:35.670596 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:36.386693 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:36.386637 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:36.482604 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:36.482578 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:36.582119 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:36.582087 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:38.139645 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:38.139610 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:43.628726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:43.628691 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:43.734537 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:43.734494 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2"] Sep 19 05:57:43.735058 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:43.735038 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerName="istio-proxy" Sep 19 05:57:43.735159 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:43.735060 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerName="istio-proxy" Sep 19 05:57:43.735228 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:43.735199 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerName="istio-proxy" Sep 19 05:57:43.735228 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:43.735212 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerName="istio-proxy" Sep 19 05:57:43.738050 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:43.738029 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2" Sep 19 05:57:43.740632 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:43.740609 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 05:57:43.745686 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:43.745650 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2"] Sep 19 05:57:43.880920 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:43.880823 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e22614-d7af-4daf-90c9-7069293455fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-875220-ljfk2\" (UID: \"f5e22614-d7af-4daf-90c9-7069293455fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2" Sep 19 05:57:43.982255 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:43.982220 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e22614-d7af-4daf-90c9-7069293455fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-875220-ljfk2\" (UID: \"f5e22614-d7af-4daf-90c9-7069293455fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2" Sep 19 05:57:43.982868 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:43.982848 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e22614-d7af-4daf-90c9-7069293455fa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-875220-ljfk2\" (UID: \"f5e22614-d7af-4daf-90c9-7069293455fa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2" Sep 19 05:57:44.050074 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:44.050034 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2" Sep 19 05:57:44.325624 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:44.325545 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2"] Sep 19 05:57:44.334177 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:57:44.334148 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5e22614_d7af_4daf_90c9_7069293455fa.slice/crio-1a635cff1dfcdfa70f8d61f28db7e16f406c2e1d3f2934435ded5fe3af5949b6 WatchSource:0}: Error finding container 1a635cff1dfcdfa70f8d61f28db7e16f406c2e1d3f2934435ded5fe3af5949b6: Status 404 returned error can't find the container with id 1a635cff1dfcdfa70f8d61f28db7e16f406c2e1d3f2934435ded5fe3af5949b6 Sep 19 05:57:44.561454 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:44.561416 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2" event={"ID":"f5e22614-d7af-4daf-90c9-7069293455fa","Type":"ContainerStarted","Data":"1a635cff1dfcdfa70f8d61f28db7e16f406c2e1d3f2934435ded5fe3af5949b6"} Sep 19 05:57:44.730708 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:44.730671 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:44.830066 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:44.830028 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:45.569998 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:45.569966 2584 generic.go:356] "Generic (PLEG): container finished" podID="f5e22614-d7af-4daf-90c9-7069293455fa" containerID="6860303196e04421b03c28ca4b5b9adc0e1ca228c9c3644c72954c9da5d47630" exitCode=0 Sep 19 05:57:45.570207 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:45.570049 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2" event={"ID":"f5e22614-d7af-4daf-90c9-7069293455fa","Type":"ContainerDied","Data":"6860303196e04421b03c28ca4b5b9adc0e1ca228c9c3644c72954c9da5d47630"} Sep 19 05:57:45.602306 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:45.602277 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:45.704147 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:45.704119 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:46.507771 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:46.507736 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:46.733786 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:46.733760 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2" Sep 19 05:57:46.813257 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:46.813175 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e22614-d7af-4daf-90c9-7069293455fa-maas-api-service-ca\") pod \"f5e22614-d7af-4daf-90c9-7069293455fa\" (UID: \"f5e22614-d7af-4daf-90c9-7069293455fa\") " Sep 19 05:57:46.813566 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:46.813543 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5e22614-d7af-4daf-90c9-7069293455fa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f5e22614-d7af-4daf-90c9-7069293455fa" (UID: "f5e22614-d7af-4daf-90c9-7069293455fa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:57:46.914728 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:46.914692 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5e22614-d7af-4daf-90c9-7069293455fa-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:57:47.211330 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:47.211299 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:47.583306 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:47.583257 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2" event={"ID":"f5e22614-d7af-4daf-90c9-7069293455fa","Type":"ContainerDied","Data":"1a635cff1dfcdfa70f8d61f28db7e16f406c2e1d3f2934435ded5fe3af5949b6"} Sep 19 05:57:47.583691 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:47.583355 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a635cff1dfcdfa70f8d61f28db7e16f406c2e1d3f2934435ded5fe3af5949b6" Sep 19 05:57:47.583691 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:47.583281 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2" Sep 19 05:57:47.992060 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:47.991976 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:48.696727 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:48.696696 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:51.155820 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.155742 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:51.220802 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.220770 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:51.440946 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.440872 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:51.691365 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.691270 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx"] Sep 19 05:57:51.691783 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.691769 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerName="istio-proxy" Sep 19 05:57:51.691831 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.691784 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4fb9a55-ffa2-41f7-9f33-6a1a9c9bbeb7" containerName="istio-proxy" Sep 19 05:57:51.691831 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.691819 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5e22614-d7af-4daf-90c9-7069293455fa" containerName="revoke-keys" Sep 19 05:57:51.691831 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.691825 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e22614-d7af-4daf-90c9-7069293455fa" containerName="revoke-keys" Sep 19 05:57:51.691924 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.691892 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5e22614-d7af-4daf-90c9-7069293455fa" containerName="revoke-keys" Sep 19 05:57:51.695262 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.695240 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.699211 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.699190 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-aec2317b-openshift-default-dockercfg-jpkjm\"" Sep 19 05:57:51.707441 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.707416 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx"] Sep 19 05:57:51.719858 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.719835 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:51.839360 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.839311 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:51.872253 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.872214 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr7pn\" (UniqueName: \"kubernetes.io/projected/1bec1b7e-4389-4562-b553-26b4bd33254d-kube-api-access-sr7pn\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.872473 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.872268 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-envoy\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.872473 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.872333 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-workload-certs\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.872473 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.872429 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-workload-socket\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.872661 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.872477 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-data\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.872661 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.872505 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-credential-socket\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.872661 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.872547 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-podinfo\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.872661 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.872587 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-token\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.872661 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.872603 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bec1b7e-4389-4562-b553-26b4bd33254d-istiod-ca-cert\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.973966 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.973867 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-envoy\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.973966 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.973915 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-workload-certs\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.973966 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.973942 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-workload-socket\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.974263 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.973987 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-data\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.974263 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.974028 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-credential-socket\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.974263 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.974076 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-podinfo\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.974263 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.974099 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-token\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.974263 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.974124 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bec1b7e-4389-4562-b553-26b4bd33254d-istiod-ca-cert\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.974263 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.974214 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7pn\" (UniqueName: \"kubernetes.io/projected/1bec1b7e-4389-4562-b553-26b4bd33254d-kube-api-access-sr7pn\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.974592 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.974374 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-workload-socket\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.974592 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.974460 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-data\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.974806 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.974780 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-workload-certs\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.974952 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.974809 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-credential-socket\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.975263 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.975012 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bec1b7e-4389-4562-b553-26b4bd33254d-istiod-ca-cert\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.976123 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.976100 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-envoy\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.976619 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.976587 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-podinfo\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.986011 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.985987 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-token\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:51.986215 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:51.986193 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr7pn\" (UniqueName: \"kubernetes.io/projected/1bec1b7e-4389-4562-b553-26b4bd33254d-kube-api-access-sr7pn\") pod \"e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:52.007201 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:52.007164 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:52.285667 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:52.285638 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx"] Sep 19 05:57:52.287544 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:57:52.287469 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bec1b7e_4389_4562_b553_26b4bd33254d.slice/crio-3e103fdb4bc0a1d7f09c5d2657bcdb5947b89964791fbb4ba27440ca7df309d9 WatchSource:0}: Error finding container 3e103fdb4bc0a1d7f09c5d2657bcdb5947b89964791fbb4ba27440ca7df309d9: Status 404 returned error can't find the container with id 3e103fdb4bc0a1d7f09c5d2657bcdb5947b89964791fbb4ba27440ca7df309d9 Sep 19 05:57:52.290720 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:52.290672 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:57:52.290832 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:52.290765 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:57:52.290898 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:52.290840 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:57:52.614353 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:52.614313 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" event={"ID":"1bec1b7e-4389-4562-b553-26b4bd33254d","Type":"ContainerStarted","Data":"0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4"} Sep 19 05:57:52.614569 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:52.614378 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" event={"ID":"1bec1b7e-4389-4562-b553-26b4bd33254d","Type":"ContainerStarted","Data":"3e103fdb4bc0a1d7f09c5d2657bcdb5947b89964791fbb4ba27440ca7df309d9"} Sep 19 05:57:52.655137 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:52.655086 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" podStartSLOduration=1.655069713 podStartE2EDuration="1.655069713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:57:52.653187426 +0000 UTC m=+1000.646414981" watchObservedRunningTime="2026-09-19 05:57:52.655069713 +0000 UTC m=+1000.648297242" Sep 19 05:57:53.007741 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:53.007658 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:53.013060 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:53.013033 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:53.619822 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:53.619793 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:53.620952 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:53.620936 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:57:54.730488 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:54.730447 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:54.757007 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:54.756950 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt"] Sep 19 05:57:54.757234 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:54.757204 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" podUID="1c052a17-232c-4bd6-8ef9-e94a8d12bc32" containerName="payload-processing" containerID="cri-o://935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e" gracePeriod=30 Sep 19 05:57:54.833505 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:54.833465 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:54.936234 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:54.936189 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:54.962663 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:54.962608 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2"] Sep 19 05:57:54.969320 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:54.969287 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-875220-ljfk2"] Sep 19 05:57:55.012967 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.012941 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:55.030368 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.030341 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" Sep 19 05:57:55.210848 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.210803 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcxpf\" (UniqueName: \"kubernetes.io/projected/1c052a17-232c-4bd6-8ef9-e94a8d12bc32-kube-api-access-lcxpf\") pod \"1c052a17-232c-4bd6-8ef9-e94a8d12bc32\" (UID: \"1c052a17-232c-4bd6-8ef9-e94a8d12bc32\") " Sep 19 05:57:55.211045 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.210868 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c052a17-232c-4bd6-8ef9-e94a8d12bc32-plugins-config-volume\") pod \"1c052a17-232c-4bd6-8ef9-e94a8d12bc32\" (UID: \"1c052a17-232c-4bd6-8ef9-e94a8d12bc32\") " Sep 19 05:57:55.211282 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.211257 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c052a17-232c-4bd6-8ef9-e94a8d12bc32-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1c052a17-232c-4bd6-8ef9-e94a8d12bc32" (UID: "1c052a17-232c-4bd6-8ef9-e94a8d12bc32"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:57:55.213089 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.213066 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c052a17-232c-4bd6-8ef9-e94a8d12bc32-kube-api-access-lcxpf" (OuterVolumeSpecName: "kube-api-access-lcxpf") pod "1c052a17-232c-4bd6-8ef9-e94a8d12bc32" (UID: "1c052a17-232c-4bd6-8ef9-e94a8d12bc32"). InnerVolumeSpecName "kube-api-access-lcxpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:57:55.311814 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.311725 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcxpf\" (UniqueName: \"kubernetes.io/projected/1c052a17-232c-4bd6-8ef9-e94a8d12bc32-kube-api-access-lcxpf\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:57:55.311814 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.311758 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c052a17-232c-4bd6-8ef9-e94a8d12bc32-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:57:55.344740 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.344710 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:55.414602 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.414573 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:55.631754 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.631721 2584 generic.go:356] "Generic (PLEG): container finished" podID="1c052a17-232c-4bd6-8ef9-e94a8d12bc32" containerID="935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e" exitCode=0 Sep 19 05:57:55.631923 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.631763 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" Sep 19 05:57:55.631923 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.631800 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" event={"ID":"1c052a17-232c-4bd6-8ef9-e94a8d12bc32","Type":"ContainerDied","Data":"935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e"} Sep 19 05:57:55.631923 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.631850 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt" event={"ID":"1c052a17-232c-4bd6-8ef9-e94a8d12bc32","Type":"ContainerDied","Data":"b3eb64dd8bcd7cb7aab77e5bafdb7083b1c17352685d37cca57f4e0f0220736e"} Sep 19 05:57:55.631923 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.631878 2584 scope.go:117] "RemoveContainer" containerID="935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e" Sep 19 05:57:55.641809 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.641790 2584 scope.go:117] "RemoveContainer" containerID="935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e" Sep 19 05:57:55.642077 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:57:55.642058 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e\": container with ID starting with 935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e not found: ID does not exist" containerID="935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e" Sep 19 05:57:55.642126 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.642086 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e"} err="failed to get container status \"935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e\": rpc error: code = NotFound desc = could not find container \"935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e\": container with ID starting with 935ec7bf038beee45424219548ea5ac93070c802667266f3c8524b87d0d1325e not found: ID does not exist" Sep 19 05:57:55.656516 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.656496 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt"] Sep 19 05:57:55.660409 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:55.660375 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-875220-c87bbc54f-gv2gt"] Sep 19 05:57:56.094282 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:56.094251 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:56.268371 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:56.268333 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c052a17-232c-4bd6-8ef9-e94a8d12bc32" path="/var/lib/kubelet/pods/1c052a17-232c-4bd6-8ef9-e94a8d12bc32/volumes" Sep 19 05:57:56.268763 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:56.268748 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5e22614-d7af-4daf-90c9-7069293455fa" path="/var/lib/kubelet/pods/f5e22614-d7af-4daf-90c9-7069293455fa/volumes" Sep 19 05:57:56.782425 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:56.782369 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:57.636091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:57.636059 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:58.414040 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:58.413991 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:58.509813 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:58.509732 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:57:59.416380 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:57:59.416342 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:00.300190 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:00.300161 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:01.207460 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:01.207418 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:02.038443 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.038392 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:02.133265 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.133225 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:02.270777 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.270746 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:02.372512 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.372481 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:02.474715 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.474684 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:02.586926 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.586887 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:02.689768 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.689688 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:02.849380 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.849346 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh"] Sep 19 05:58:02.849841 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.849828 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c052a17-232c-4bd6-8ef9-e94a8d12bc32" containerName="payload-processing" Sep 19 05:58:02.849887 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.849843 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c052a17-232c-4bd6-8ef9-e94a8d12bc32" containerName="payload-processing" Sep 19 05:58:02.849949 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.849939 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c052a17-232c-4bd6-8ef9-e94a8d12bc32" containerName="payload-processing" Sep 19 05:58:02.853148 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.853124 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" Sep 19 05:58:02.855915 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.855893 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-aec2317b\"" Sep 19 05:58:02.856048 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.855961 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-aec2317b-dockercfg-d75hh\"" Sep 19 05:58:02.864191 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.864167 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh"] Sep 19 05:58:02.890128 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.890095 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttd75\" (UniqueName: \"kubernetes.io/projected/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0-kube-api-access-ttd75\") pod \"payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh\" (UID: \"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0\") " pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" Sep 19 05:58:02.890342 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.890205 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0-plugins-config-volume\") pod \"payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh\" (UID: \"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0\") " pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" Sep 19 05:58:02.991260 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.991169 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttd75\" (UniqueName: \"kubernetes.io/projected/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0-kube-api-access-ttd75\") pod \"payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh\" (UID: \"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0\") " pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" Sep 19 05:58:02.991489 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.991283 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0-plugins-config-volume\") pod \"payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh\" (UID: \"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0\") " pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" Sep 19 05:58:02.991945 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.991927 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0-plugins-config-volume\") pod \"payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh\" (UID: \"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0\") " pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" Sep 19 05:58:02.999333 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:02.999309 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttd75\" (UniqueName: \"kubernetes.io/projected/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0-kube-api-access-ttd75\") pod \"payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh\" (UID: \"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0\") " pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" Sep 19 05:58:03.127610 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:03.127579 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:03.164713 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:03.164679 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" Sep 19 05:58:03.439741 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:03.439707 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh"] Sep 19 05:58:03.443877 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:58:03.443848 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe5e72ff_1c7e_486e_a0a6_8e166f3e74a0.slice/crio-1a410367c7f0acb602b87dc5fda786d4100f0c968ede0f0527fa56336812270c WatchSource:0}: Error finding container 1a410367c7f0acb602b87dc5fda786d4100f0c968ede0f0527fa56336812270c: Status 404 returned error can't find the container with id 1a410367c7f0acb602b87dc5fda786d4100f0c968ede0f0527fa56336812270c Sep 19 05:58:03.450444 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:03.450384 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:03.682233 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:03.682192 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" event={"ID":"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0","Type":"ContainerStarted","Data":"4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1"} Sep 19 05:58:03.682464 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:03.682253 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" event={"ID":"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0","Type":"ContainerStarted","Data":"1a410367c7f0acb602b87dc5fda786d4100f0c968ede0f0527fa56336812270c"} Sep 19 05:58:03.682464 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:03.682281 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" Sep 19 05:58:03.701624 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:03.701521 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" podStartSLOduration=1.70150363 podStartE2EDuration="1.70150363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:58:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:58:03.699139169 +0000 UTC m=+1011.692366699" watchObservedRunningTime="2026-09-19 05:58:03.70150363 +0000 UTC m=+1011.694731161" Sep 19 05:58:03.729729 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:03.729702 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:04.700003 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:04.699971 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:04.822208 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:04.822162 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:04.971802 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:04.971711 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:14.690389 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:14.690360 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" Sep 19 05:58:14.837203 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:14.837172 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:25.125145 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:25.125082 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:25.241899 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:25.241864 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:26.007574 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:26.007525 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:26.816820 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:26.816771 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:27.593561 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:27.593523 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:28.332501 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:28.332446 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:28.525218 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:28.525184 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:29.050282 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:29.050240 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:29.141022 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:29.140981 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:29.494548 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:29.494518 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:29.590739 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:29.590688 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:32.868645 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:32.868609 2584 generic.go:356] "Generic (PLEG): container finished" podID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerID="0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4" exitCode=0 Sep 19 05:58:32.869102 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:32.868684 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" event={"ID":"1bec1b7e-4389-4562-b553-26b4bd33254d","Type":"ContainerDied","Data":"0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4"} Sep 19 05:58:32.869148 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:32.869103 2584 scope.go:117] "RemoveContainer" containerID="0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4" Sep 19 05:58:32.870283 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:32.870253 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:58:32.870336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:32.870325 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:58:32.870392 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:32.870380 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:58:33.875994 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:33.875958 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" event={"ID":"1bec1b7e-4389-4562-b553-26b4bd33254d","Type":"ContainerStarted","Data":"2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1"} Sep 19 05:58:34.009057 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:34.009010 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:58:34.009263 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:34.009150 2584 patch_prober.go:28] interesting pod/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 19 05:58:34.009263 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:34.009197 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 19 05:58:35.021982 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:35.021924 2584 patch_prober.go:28] interesting pod/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 19 05:58:35.022523 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:35.022023 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 19 05:58:35.219147 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:35.219082 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:35.318717 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:35.318605 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:35.600649 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:35.600612 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc"] Sep 19 05:58:35.603310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:35.603293 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc" Sep 19 05:58:35.605567 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:35.605543 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 05:58:35.611298 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:35.611271 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc"] Sep 19 05:58:35.643495 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:35.643456 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fe22d8-fcc1-4853-a644-89423871b4e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc\" (UID: \"a8fe22d8-fcc1-4853-a644-89423871b4e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc" Sep 19 05:58:35.745149 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:35.745111 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fe22d8-fcc1-4853-a644-89423871b4e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc\" (UID: \"a8fe22d8-fcc1-4853-a644-89423871b4e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc" Sep 19 05:58:35.745794 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:35.745772 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fe22d8-fcc1-4853-a644-89423871b4e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc\" (UID: \"a8fe22d8-fcc1-4853-a644-89423871b4e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc" Sep 19 05:58:35.918160 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:35.918055 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc" Sep 19 05:58:36.033449 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:36.013813 2584 patch_prober.go:28] interesting pod/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 19 05:58:36.033449 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:36.013879 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 19 05:58:36.197662 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:36.197624 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc"] Sep 19 05:58:36.199355 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:58:36.199329 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8fe22d8_fcc1_4853_a644_89423871b4e2.slice/crio-17093c68c241e8ed61b22346779e47bd28309f99fa3088e7f083c7d15b2f6f40 WatchSource:0}: Error finding container 17093c68c241e8ed61b22346779e47bd28309f99fa3088e7f083c7d15b2f6f40: Status 404 returned error can't find the container with id 17093c68c241e8ed61b22346779e47bd28309f99fa3088e7f083c7d15b2f6f40 Sep 19 05:58:36.423589 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:36.423550 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:36.895336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:36.895249 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc" event={"ID":"a8fe22d8-fcc1-4853-a644-89423871b4e2","Type":"ContainerStarted","Data":"98c8df9a5f54a1520f011feb608f9983a57cf3ca213faa19952264b9454ab10f"} Sep 19 05:58:36.895336 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:36.895300 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc" event={"ID":"a8fe22d8-fcc1-4853-a644-89423871b4e2","Type":"ContainerStarted","Data":"17093c68c241e8ed61b22346779e47bd28309f99fa3088e7f083c7d15b2f6f40"} Sep 19 05:58:37.007960 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:37.007920 2584 patch_prober.go:28] interesting pod/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 19 05:58:37.008172 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:37.007990 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 19 05:58:37.403955 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:37.403924 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:37.903136 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:37.903099 2584 generic.go:356] "Generic (PLEG): container finished" podID="a8fe22d8-fcc1-4853-a644-89423871b4e2" containerID="98c8df9a5f54a1520f011feb608f9983a57cf3ca213faa19952264b9454ab10f" exitCode=0 Sep 19 05:58:37.903318 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:37.903189 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc" event={"ID":"a8fe22d8-fcc1-4853-a644-89423871b4e2","Type":"ContainerDied","Data":"98c8df9a5f54a1520f011feb608f9983a57cf3ca213faa19952264b9454ab10f"} Sep 19 05:58:38.008556 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:38.008502 2584 patch_prober.go:28] interesting pod/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Sep 19 05:58:38.008760 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:38.008585 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Sep 19 05:58:38.242945 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:38.242855 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:38.478787 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:38.478756 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:38.620709 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:38.620663 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:58:39.015749 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:39.015283 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:58:39.017624 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:39.017586 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:58:39.074143 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:39.074111 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc" Sep 19 05:58:39.128738 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:39.128707 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:39.182080 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:39.181995 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fe22d8-fcc1-4853-a644-89423871b4e2-maas-api-service-ca\") pod \"a8fe22d8-fcc1-4853-a644-89423871b4e2\" (UID: \"a8fe22d8-fcc1-4853-a644-89423871b4e2\") " Sep 19 05:58:39.182353 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:39.182328 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8fe22d8-fcc1-4853-a644-89423871b4e2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a8fe22d8-fcc1-4853-a644-89423871b4e2" (UID: "a8fe22d8-fcc1-4853-a644-89423871b4e2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:58:39.283176 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:39.283138 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fe22d8-fcc1-4853-a644-89423871b4e2-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:58:39.916326 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:39.916286 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc" Sep 19 05:58:39.916836 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:39.916282 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc" event={"ID":"a8fe22d8-fcc1-4853-a644-89423871b4e2","Type":"ContainerDied","Data":"17093c68c241e8ed61b22346779e47bd28309f99fa3088e7f083c7d15b2f6f40"} Sep 19 05:58:39.916836 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:39.916517 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17093c68c241e8ed61b22346779e47bd28309f99fa3088e7f083c7d15b2f6f40" Sep 19 05:58:39.956622 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:39.956591 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:40.679906 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:40.679862 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:46.188868 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.188832 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:46.223005 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.222969 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh"] Sep 19 05:58:46.223529 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.223499 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" podUID="fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0" containerName="payload-processing" containerID="cri-o://4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1" gracePeriod=30 Sep 19 05:58:46.289359 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.289328 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:46.391150 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.391101 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc"] Sep 19 05:58:46.392792 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.392737 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:46.394463 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.394438 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec2317b-q2kcc"] Sep 19 05:58:46.489449 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.489421 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:46.499411 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.499371 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" Sep 19 05:58:46.557213 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.557182 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttd75\" (UniqueName: \"kubernetes.io/projected/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0-kube-api-access-ttd75\") pod \"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0\" (UID: \"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0\") " Sep 19 05:58:46.557390 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.557242 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0-plugins-config-volume\") pod \"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0\" (UID: \"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0\") " Sep 19 05:58:46.557756 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.557728 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0" (UID: "fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:58:46.559388 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.559364 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0-kube-api-access-ttd75" (OuterVolumeSpecName: "kube-api-access-ttd75") pod "fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0" (UID: "fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0"). InnerVolumeSpecName "kube-api-access-ttd75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:58:46.658360 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.658319 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttd75\" (UniqueName: \"kubernetes.io/projected/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0-kube-api-access-ttd75\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:58:46.658360 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.658353 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:58:46.764655 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.764568 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:46.913281 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.913252 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:46.958976 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.958940 2584 generic.go:356] "Generic (PLEG): container finished" podID="fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0" containerID="4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1" exitCode=0 Sep 19 05:58:46.959126 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.958989 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" Sep 19 05:58:46.959126 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.959025 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" event={"ID":"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0","Type":"ContainerDied","Data":"4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1"} Sep 19 05:58:46.959126 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.959099 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh" event={"ID":"fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0","Type":"ContainerDied","Data":"1a410367c7f0acb602b87dc5fda786d4100f0c968ede0f0527fa56336812270c"} Sep 19 05:58:46.959264 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.959135 2584 scope.go:117] "RemoveContainer" containerID="4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1" Sep 19 05:58:46.969584 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.969566 2584 scope.go:117] "RemoveContainer" containerID="4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1" Sep 19 05:58:46.969871 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:58:46.969853 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1\": container with ID starting with 4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1 not found: ID does not exist" containerID="4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1" Sep 19 05:58:46.969931 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.969880 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1"} err="failed to get container status \"4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1\": rpc error: code = NotFound desc = could not find container \"4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1\": container with ID starting with 4c4742fda035700371dcd80d7f1e35a849a898cee533cd691ab1318a88cf76e1 not found: ID does not exist" Sep 19 05:58:46.985728 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.985696 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh"] Sep 19 05:58:46.987771 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:46.987750 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aec2317b-659fc99cfc-2flhh"] Sep 19 05:58:47.617961 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:47.617924 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:48.267710 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:48.267668 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8fe22d8-fcc1-4853-a644-89423871b4e2" path="/var/lib/kubelet/pods/a8fe22d8-fcc1-4853-a644-89423871b4e2/volumes" Sep 19 05:58:48.268062 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:48.268045 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0" path="/var/lib/kubelet/pods/fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0/volumes" Sep 19 05:58:48.478630 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:48.478591 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:49.200542 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:49.200499 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:50.146363 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:50.146329 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:52.416565 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:52.416529 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:52.517099 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:52.517065 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:52.673199 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:52.673115 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx"] Sep 19 05:58:52.673390 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:52.673365 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" containerID="cri-o://2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1" gracePeriod=30 Sep 19 05:58:52.699021 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:52.698992 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:53.047775 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:53.047688 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:53.165363 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:53.165330 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:53.294567 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:53.294535 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:54.019235 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:54.019194 2584 patch_prober.go:28] interesting pod/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 19 05:58:54.019631 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:54.019266 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 19 05:58:57.563381 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:57.563294 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:57.932309 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:57.932280 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:58:58.024717 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.024683 2584 generic.go:356] "Generic (PLEG): container finished" podID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerID="2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1" exitCode=0 Sep 19 05:58:58.024917 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.024742 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" Sep 19 05:58:58.024917 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.024776 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" event={"ID":"1bec1b7e-4389-4562-b553-26b4bd33254d","Type":"ContainerDied","Data":"2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1"} Sep 19 05:58:58.024917 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.024833 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx" event={"ID":"1bec1b7e-4389-4562-b553-26b4bd33254d","Type":"ContainerDied","Data":"3e103fdb4bc0a1d7f09c5d2657bcdb5947b89964791fbb4ba27440ca7df309d9"} Sep 19 05:58:58.024917 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.024858 2584 scope.go:117] "RemoveContainer" containerID="2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1" Sep 19 05:58:58.034173 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.034156 2584 scope.go:117] "RemoveContainer" containerID="0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4" Sep 19 05:58:58.042899 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.042880 2584 scope.go:117] "RemoveContainer" containerID="2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1" Sep 19 05:58:58.043235 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:58:58.043214 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1\": container with ID starting with 2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1 not found: ID does not exist" containerID="2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1" Sep 19 05:58:58.043280 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.043246 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1"} err="failed to get container status \"2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1\": rpc error: code = NotFound desc = could not find container \"2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1\": container with ID starting with 2267aba5d7feed4b9d39aa54fa69f1881dc612bd1a5235b4060dabf7feb17eb1 not found: ID does not exist" Sep 19 05:58:58.043280 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.043265 2584 scope.go:117] "RemoveContainer" containerID="0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4" Sep 19 05:58:58.043550 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:58:58.043534 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4\": container with ID starting with 0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4 not found: ID does not exist" containerID="0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4" Sep 19 05:58:58.043598 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.043557 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4"} err="failed to get container status \"0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4\": rpc error: code = NotFound desc = could not find container \"0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4\": container with ID starting with 0903cf042b6e200f234a920b5745096ccb88efaedcb7bb6a01914fa88dbdedc4 not found: ID does not exist" Sep 19 05:58:58.077107 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077067 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-credential-socket\") pod \"1bec1b7e-4389-4562-b553-26b4bd33254d\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " Sep 19 05:58:58.077107 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077108 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bec1b7e-4389-4562-b553-26b4bd33254d-istiod-ca-cert\") pod \"1bec1b7e-4389-4562-b553-26b4bd33254d\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " Sep 19 05:58:58.077287 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077205 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr7pn\" (UniqueName: \"kubernetes.io/projected/1bec1b7e-4389-4562-b553-26b4bd33254d-kube-api-access-sr7pn\") pod \"1bec1b7e-4389-4562-b553-26b4bd33254d\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " Sep 19 05:58:58.077287 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077243 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-data\") pod \"1bec1b7e-4389-4562-b553-26b4bd33254d\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " Sep 19 05:58:58.077287 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077268 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-podinfo\") pod \"1bec1b7e-4389-4562-b553-26b4bd33254d\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " Sep 19 05:58:58.077447 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077292 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-token\") pod \"1bec1b7e-4389-4562-b553-26b4bd33254d\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " Sep 19 05:58:58.077447 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077330 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-workload-socket\") pod \"1bec1b7e-4389-4562-b553-26b4bd33254d\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " Sep 19 05:58:58.077447 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077363 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-envoy\") pod \"1bec1b7e-4389-4562-b553-26b4bd33254d\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " Sep 19 05:58:58.077447 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077435 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1bec1b7e-4389-4562-b553-26b4bd33254d" (UID: "1bec1b7e-4389-4562-b553-26b4bd33254d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 05:58:58.077665 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077446 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-workload-certs\") pod \"1bec1b7e-4389-4562-b553-26b4bd33254d\" (UID: \"1bec1b7e-4389-4562-b553-26b4bd33254d\") " Sep 19 05:58:58.077665 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077631 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-data" (OuterVolumeSpecName: "istio-data") pod "1bec1b7e-4389-4562-b553-26b4bd33254d" (UID: "1bec1b7e-4389-4562-b553-26b4bd33254d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 05:58:58.077665 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077647 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bec1b7e-4389-4562-b553-26b4bd33254d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1bec1b7e-4389-4562-b553-26b4bd33254d" (UID: "1bec1b7e-4389-4562-b553-26b4bd33254d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:58:58.077850 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077825 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1bec1b7e-4389-4562-b553-26b4bd33254d" (UID: "1bec1b7e-4389-4562-b553-26b4bd33254d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 05:58:58.077850 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077838 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1bec1b7e-4389-4562-b553-26b4bd33254d" (UID: "1bec1b7e-4389-4562-b553-26b4bd33254d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 05:58:58.077964 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077912 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-workload-certs\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:58:58.077964 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077929 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-credential-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:58:58.077964 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077947 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bec1b7e-4389-4562-b553-26b4bd33254d-istiod-ca-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:58:58.078109 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077965 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-data\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:58:58.078109 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.077977 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-workload-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:58:58.079767 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.079743 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1bec1b7e-4389-4562-b553-26b4bd33254d" (UID: "1bec1b7e-4389-4562-b553-26b4bd33254d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 05:58:58.079882 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.079743 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1bec1b7e-4389-4562-b553-26b4bd33254d" (UID: "1bec1b7e-4389-4562-b553-26b4bd33254d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 05:58:58.079882 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.079769 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-token" (OuterVolumeSpecName: "istio-token") pod "1bec1b7e-4389-4562-b553-26b4bd33254d" (UID: "1bec1b7e-4389-4562-b553-26b4bd33254d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:58:58.079882 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.079789 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bec1b7e-4389-4562-b553-26b4bd33254d-kube-api-access-sr7pn" (OuterVolumeSpecName: "kube-api-access-sr7pn") pod "1bec1b7e-4389-4562-b553-26b4bd33254d" (UID: "1bec1b7e-4389-4562-b553-26b4bd33254d"). InnerVolumeSpecName "kube-api-access-sr7pn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:58:58.178719 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.178635 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-envoy\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:58:58.178719 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.178668 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr7pn\" (UniqueName: \"kubernetes.io/projected/1bec1b7e-4389-4562-b553-26b4bd33254d-kube-api-access-sr7pn\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:58:58.178719 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.178679 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-podinfo\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:58:58.178719 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.178689 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bec1b7e-4389-4562-b553-26b4bd33254d-istio-token\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:58:58.343089 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.343052 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx"] Sep 19 05:58:58.344878 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.344853 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:58.345280 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:58.345255 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-aec2317b-openshift-default-7cf96449d4-hlfpx"] Sep 19 05:58:59.127791 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:59.127757 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:59.233092 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:59.233064 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:59.327610 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:59.327579 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:58:59.520256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:58:59.520173 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:00.078595 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:00.078561 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:00.268047 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:00.268014 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" path="/var/lib/kubelet/pods/1bec1b7e-4389-4562-b553-26b4bd33254d/volumes" Sep 19 05:59:02.182205 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:02.182177 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:05.276614 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:05.276582 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:05.349300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:05.349269 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:06.241699 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.241660 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:06.532702 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.532621 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9"] Sep 19 05:59:06.533094 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.533081 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" Sep 19 05:59:06.533145 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.533096 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" Sep 19 05:59:06.533145 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.533107 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0" containerName="payload-processing" Sep 19 05:59:06.533145 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.533113 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0" containerName="payload-processing" Sep 19 05:59:06.533145 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.533125 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" Sep 19 05:59:06.533145 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.533131 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" Sep 19 05:59:06.533310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.533151 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8fe22d8-fcc1-4853-a644-89423871b4e2" containerName="revoke-keys" Sep 19 05:59:06.533310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.533157 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8fe22d8-fcc1-4853-a644-89423871b4e2" containerName="revoke-keys" Sep 19 05:59:06.533310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.533236 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" Sep 19 05:59:06.533310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.533245 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bec1b7e-4389-4562-b553-26b4bd33254d" containerName="istio-proxy" Sep 19 05:59:06.533310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.533250 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8fe22d8-fcc1-4853-a644-89423871b4e2" containerName="revoke-keys" Sep 19 05:59:06.533310 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.533261 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe5e72ff-1c7e-486e-a0a6-8e166f3e74a0" containerName="payload-processing" Sep 19 05:59:06.535975 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.535960 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.538601 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.538579 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-2b3fb3-openshift-default-dockercfg-h9bv4\"" Sep 19 05:59:06.551133 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.551107 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9"] Sep 19 05:59:06.569688 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.569656 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:06.570806 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.570777 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-istio-envoy\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.570939 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.570830 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/195b28ee-0890-40b9-854f-c8a507da24a1-istio-podinfo\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.570939 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.570897 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-credential-socket\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.570939 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.570924 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-workload-certs\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.571128 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.570982 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-istio-data\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.571128 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.571008 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wsqf\" (UniqueName: \"kubernetes.io/projected/195b28ee-0890-40b9-854f-c8a507da24a1-kube-api-access-9wsqf\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.571128 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.571047 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/195b28ee-0890-40b9-854f-c8a507da24a1-istio-token\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.571128 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.571086 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/195b28ee-0890-40b9-854f-c8a507da24a1-istiod-ca-cert\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.571341 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.571145 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-workload-socket\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.671865 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.671826 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-istio-envoy\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672046 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.671874 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/195b28ee-0890-40b9-854f-c8a507da24a1-istio-podinfo\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672046 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.671910 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-credential-socket\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672046 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.671934 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-workload-certs\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672046 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.671987 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-istio-data\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672046 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.672011 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wsqf\" (UniqueName: \"kubernetes.io/projected/195b28ee-0890-40b9-854f-c8a507da24a1-kube-api-access-9wsqf\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672301 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.672049 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/195b28ee-0890-40b9-854f-c8a507da24a1-istio-token\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672301 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.672091 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/195b28ee-0890-40b9-854f-c8a507da24a1-istiod-ca-cert\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672301 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.672140 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-workload-socket\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672511 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.672485 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-istio-data\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672580 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.672505 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-workload-certs\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672625 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.672581 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-workload-socket\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672625 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.672595 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-credential-socket\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.672940 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.672919 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/195b28ee-0890-40b9-854f-c8a507da24a1-istiod-ca-cert\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.674300 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.674282 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-istio-envoy\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.674707 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.674692 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/195b28ee-0890-40b9-854f-c8a507da24a1-istio-podinfo\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.680359 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.680327 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/195b28ee-0890-40b9-854f-c8a507da24a1-istio-token\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.680501 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.680484 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wsqf\" (UniqueName: \"kubernetes.io/projected/195b28ee-0890-40b9-854f-c8a507da24a1-kube-api-access-9wsqf\") pod \"e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:06.776596 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.776550 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:06.846244 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:06.846211 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:07.125133 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:07.125105 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9"] Sep 19 05:59:07.127032 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:59:07.127000 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod195b28ee_0890_40b9_854f_c8a507da24a1.slice/crio-f8200b5d1aeb8cee5059aeda47e61d0e49ffb7ddb4f51282849517bdb9fd1ee6 WatchSource:0}: Error finding container f8200b5d1aeb8cee5059aeda47e61d0e49ffb7ddb4f51282849517bdb9fd1ee6: Status 404 returned error can't find the container with id f8200b5d1aeb8cee5059aeda47e61d0e49ffb7ddb4f51282849517bdb9fd1ee6 Sep 19 05:59:07.129477 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:07.129439 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:59:07.129558 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:07.129532 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:59:07.129608 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:07.129580 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:59:08.087455 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:08.087424 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" event={"ID":"195b28ee-0890-40b9-854f-c8a507da24a1","Type":"ContainerStarted","Data":"6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b"} Sep 19 05:59:08.087841 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:08.087473 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" event={"ID":"195b28ee-0890-40b9-854f-c8a507da24a1","Type":"ContainerStarted","Data":"f8200b5d1aeb8cee5059aeda47e61d0e49ffb7ddb4f51282849517bdb9fd1ee6"} Sep 19 05:59:08.110714 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:08.110658 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" podStartSLOduration=2.110641524 podStartE2EDuration="2.110641524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:59:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:59:08.108993587 +0000 UTC m=+1076.102221129" watchObservedRunningTime="2026-09-19 05:59:08.110641524 +0000 UTC m=+1076.103869116" Sep 19 05:59:08.846632 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:08.846592 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:08.851558 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:08.851531 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:09.093663 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:09.093633 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:09.094566 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:09.094542 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:09.399272 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:09.399243 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:10.143834 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:10.143806 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:10.243978 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:10.243944 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:10.484641 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:10.484553 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:11.217000 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:11.216970 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:12.108713 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:12.108686 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:12.644649 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:12.644615 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:12.857722 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:12.857691 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:12.959247 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:12.959156 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:13.062612 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:13.062578 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:13.709677 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:13.709648 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:13.777458 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:13.777389 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:13.975705 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:13.975615 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:14.289822 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.289741 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:14.384479 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.384445 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:14.429461 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.429423 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5"] Sep 19 05:59:14.438249 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.438209 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" Sep 19 05:59:14.441660 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.441628 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-2b3fb3-dockercfg-tjnjw\"" Sep 19 05:59:14.441921 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.441901 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-2b3fb3\"" Sep 19 05:59:14.443609 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.443578 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5"] Sep 19 05:59:14.549186 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.549089 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a776fb4a-b357-4054-b367-ded97f53b90d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5\" (UID: \"a776fb4a-b357-4054-b367-ded97f53b90d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" Sep 19 05:59:14.549389 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.549298 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs5cq\" (UniqueName: \"kubernetes.io/projected/a776fb4a-b357-4054-b367-ded97f53b90d-kube-api-access-cs5cq\") pod \"payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5\" (UID: \"a776fb4a-b357-4054-b367-ded97f53b90d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" Sep 19 05:59:14.650199 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.650158 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs5cq\" (UniqueName: \"kubernetes.io/projected/a776fb4a-b357-4054-b367-ded97f53b90d-kube-api-access-cs5cq\") pod \"payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5\" (UID: \"a776fb4a-b357-4054-b367-ded97f53b90d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" Sep 19 05:59:14.650378 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.650218 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a776fb4a-b357-4054-b367-ded97f53b90d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5\" (UID: \"a776fb4a-b357-4054-b367-ded97f53b90d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" Sep 19 05:59:14.650934 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.650914 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a776fb4a-b357-4054-b367-ded97f53b90d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5\" (UID: \"a776fb4a-b357-4054-b367-ded97f53b90d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" Sep 19 05:59:14.658186 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.658163 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs5cq\" (UniqueName: \"kubernetes.io/projected/a776fb4a-b357-4054-b367-ded97f53b90d-kube-api-access-cs5cq\") pod \"payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5\" (UID: \"a776fb4a-b357-4054-b367-ded97f53b90d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" Sep 19 05:59:14.751530 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:14.751495 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" Sep 19 05:59:15.025898 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:15.025869 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5"] Sep 19 05:59:15.027568 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:59:15.027541 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda776fb4a_b357_4054_b367_ded97f53b90d.slice/crio-d10c5e7d87bc25a8eb8f2304293e77999f01b9a2fa789ca9b06db281aa207f28 WatchSource:0}: Error finding container d10c5e7d87bc25a8eb8f2304293e77999f01b9a2fa789ca9b06db281aa207f28: Status 404 returned error can't find the container with id d10c5e7d87bc25a8eb8f2304293e77999f01b9a2fa789ca9b06db281aa207f28 Sep 19 05:59:15.131175 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:15.131137 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" event={"ID":"a776fb4a-b357-4054-b367-ded97f53b90d","Type":"ContainerStarted","Data":"9323d6833466799ccad972dcf1f05f2fe2b4feaff237d2f78e008692e8128fad"} Sep 19 05:59:15.131369 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:15.131201 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" Sep 19 05:59:15.131369 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:15.131225 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" event={"ID":"a776fb4a-b357-4054-b367-ded97f53b90d","Type":"ContainerStarted","Data":"d10c5e7d87bc25a8eb8f2304293e77999f01b9a2fa789ca9b06db281aa207f28"} Sep 19 05:59:15.151135 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:15.151073 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" podStartSLOduration=1.151054659 podStartE2EDuration="1.151054659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:59:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:59:15.149816317 +0000 UTC m=+1083.143043852" watchObservedRunningTime="2026-09-19 05:59:15.151054659 +0000 UTC m=+1083.144282189" Sep 19 05:59:18.008174 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:18.008144 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:19.904458 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:19.904425 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:24.846608 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:24.846570 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:24.947951 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:24.947914 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:26.138976 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:26.138948 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" Sep 19 05:59:35.298676 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:35.298642 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:36.911424 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:36.911371 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:37.618846 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:37.618806 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:37.621123 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:37.621098 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4"] Sep 19 05:59:37.625528 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:37.625504 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4" Sep 19 05:59:37.628018 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:37.627997 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 05:59:37.636198 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:37.636168 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4"] Sep 19 05:59:37.693837 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:37.693794 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a245b83-853c-4c3f-af6a-d308cd02da06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4\" (UID: \"9a245b83-853c-4c3f-af6a-d308cd02da06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4" Sep 19 05:59:37.794646 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:37.794599 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a245b83-853c-4c3f-af6a-d308cd02da06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4\" (UID: \"9a245b83-853c-4c3f-af6a-d308cd02da06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4" Sep 19 05:59:37.795225 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:37.795204 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a245b83-853c-4c3f-af6a-d308cd02da06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4\" (UID: \"9a245b83-853c-4c3f-af6a-d308cd02da06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4" Sep 19 05:59:37.940548 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:37.940446 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4" Sep 19 05:59:38.224548 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:38.224502 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4"] Sep 19 05:59:38.226857 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:59:38.226826 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a245b83_853c_4c3f_af6a_d308cd02da06.slice/crio-ff4f4cb538f8938296830f08e5e0bc69f1c4d3bcdd6faa471873067793f1102c WatchSource:0}: Error finding container ff4f4cb538f8938296830f08e5e0bc69f1c4d3bcdd6faa471873067793f1102c: Status 404 returned error can't find the container with id ff4f4cb538f8938296830f08e5e0bc69f1c4d3bcdd6faa471873067793f1102c Sep 19 05:59:38.280939 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:38.280909 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4" event={"ID":"9a245b83-853c-4c3f-af6a-d308cd02da06","Type":"ContainerStarted","Data":"ff4f4cb538f8938296830f08e5e0bc69f1c4d3bcdd6faa471873067793f1102c"} Sep 19 05:59:38.700818 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:38.700788 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:39.288956 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:39.288917 2584 generic.go:356] "Generic (PLEG): container finished" podID="9a245b83-853c-4c3f-af6a-d308cd02da06" containerID="6d0b10c01c3d63787d4048fa918b1d5df7c523550db0239764fc7ab01fc1eb65" exitCode=0 Sep 19 05:59:39.289467 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:39.288997 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4" event={"ID":"9a245b83-853c-4c3f-af6a-d308cd02da06","Type":"ContainerDied","Data":"6d0b10c01c3d63787d4048fa918b1d5df7c523550db0239764fc7ab01fc1eb65"} Sep 19 05:59:39.399217 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:39.399134 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:40.089925 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.089897 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:40.135575 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.135535 2584 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 19 05:59:40.135783 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.135608 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 19 05:59:40.199154 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.199119 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:40.296780 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.296747 2584 generic.go:356] "Generic (PLEG): container finished" podID="195b28ee-0890-40b9-854f-c8a507da24a1" containerID="6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b" exitCode=0 Sep 19 05:59:40.297164 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.296816 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" event={"ID":"195b28ee-0890-40b9-854f-c8a507da24a1","Type":"ContainerDied","Data":"6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b"} Sep 19 05:59:40.297551 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.297431 2584 scope.go:117] "RemoveContainer" containerID="6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b" Sep 19 05:59:40.298606 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.298570 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:59:40.298726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.298640 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:59:40.298726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.298677 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 05:59:40.464464 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.464435 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4" Sep 19 05:59:40.524479 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.524445 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a245b83-853c-4c3f-af6a-d308cd02da06-maas-api-service-ca\") pod \"9a245b83-853c-4c3f-af6a-d308cd02da06\" (UID: \"9a245b83-853c-4c3f-af6a-d308cd02da06\") " Sep 19 05:59:40.524880 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.524855 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a245b83-853c-4c3f-af6a-d308cd02da06-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9a245b83-853c-4c3f-af6a-d308cd02da06" (UID: "9a245b83-853c-4c3f-af6a-d308cd02da06"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:59:40.536722 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.536693 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:40.625823 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.625738 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a245b83-853c-4c3f-af6a-d308cd02da06-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:59:40.875618 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.875580 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:40.976632 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:40.976551 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:41.077088 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:41.077049 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:41.305482 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:41.305384 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" event={"ID":"195b28ee-0890-40b9-854f-c8a507da24a1","Type":"ContainerStarted","Data":"1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37"} Sep 19 05:59:41.306906 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:41.306876 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4" Sep 19 05:59:41.307048 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:41.306914 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4" event={"ID":"9a245b83-853c-4c3f-af6a-d308cd02da06","Type":"ContainerDied","Data":"ff4f4cb538f8938296830f08e5e0bc69f1c4d3bcdd6faa471873067793f1102c"} Sep 19 05:59:41.307048 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:41.306954 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff4f4cb538f8938296830f08e5e0bc69f1c4d3bcdd6faa471873067793f1102c" Sep 19 05:59:41.797539 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:41.797505 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:41.847393 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:41.847356 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:41.847637 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:41.847614 2584 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 19 05:59:41.847685 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:41.847663 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 19 05:59:42.786242 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:42.786214 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:42.846970 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:42.846928 2584 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 19 05:59:42.847145 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:42.847003 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 19 05:59:43.625994 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:43.625942 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:43.847134 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:43.847097 2584 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 19 05:59:43.847585 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:43.847153 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 19 05:59:44.024225 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:44.024135 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:44.125251 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:44.125207 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:44.814539 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:44.814504 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:44.846796 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:44.846755 2584 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 19 05:59:44.846968 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:44.846816 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 19 05:59:45.613185 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:45.613149 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:45.847159 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:45.847120 2584 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 19 05:59:45.847343 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:45.847198 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 19 05:59:46.364343 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:46.364311 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:46.847256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:46.847223 2584 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Sep 19 05:59:46.847680 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:46.847277 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Sep 19 05:59:47.305630 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:47.305541 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:47.850691 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:47.850656 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:47.851165 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:47.851059 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:47.851726 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:47.851703 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 05:59:48.012254 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:48.012215 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:48.811513 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:48.811481 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:48.911827 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:48.911792 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:49.075472 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.075355 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:49.395957 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.395926 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:49.483540 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.483509 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:49.484989 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.484969 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj"] Sep 19 05:59:49.485455 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.485439 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a245b83-853c-4c3f-af6a-d308cd02da06" containerName="revoke-keys" Sep 19 05:59:49.485455 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.485453 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a245b83-853c-4c3f-af6a-d308cd02da06" containerName="revoke-keys" Sep 19 05:59:49.485559 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.485551 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a245b83-853c-4c3f-af6a-d308cd02da06" containerName="revoke-keys" Sep 19 05:59:49.488788 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.488770 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" Sep 19 05:59:49.492158 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.492140 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-836c16-dockercfg-8znf6\"" Sep 19 05:59:49.493144 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.493127 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-836c16\"" Sep 19 05:59:49.508495 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.508462 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj"] Sep 19 05:59:49.618202 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.618168 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0d1284-2493-4e0e-be3e-771d73140c45-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj\" (UID: \"7d0d1284-2493-4e0e-be3e-771d73140c45\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" Sep 19 05:59:49.618381 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.618316 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz8xw\" (UniqueName: \"kubernetes.io/projected/7d0d1284-2493-4e0e-be3e-771d73140c45-kube-api-access-xz8xw\") pod \"payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj\" (UID: \"7d0d1284-2493-4e0e-be3e-771d73140c45\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" Sep 19 05:59:49.719256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.719154 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz8xw\" (UniqueName: \"kubernetes.io/projected/7d0d1284-2493-4e0e-be3e-771d73140c45-kube-api-access-xz8xw\") pod \"payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj\" (UID: \"7d0d1284-2493-4e0e-be3e-771d73140c45\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" Sep 19 05:59:49.719256 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.719252 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0d1284-2493-4e0e-be3e-771d73140c45-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj\" (UID: \"7d0d1284-2493-4e0e-be3e-771d73140c45\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" Sep 19 05:59:49.720076 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.720054 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0d1284-2493-4e0e-be3e-771d73140c45-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj\" (UID: \"7d0d1284-2493-4e0e-be3e-771d73140c45\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" Sep 19 05:59:49.726825 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.726792 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz8xw\" (UniqueName: \"kubernetes.io/projected/7d0d1284-2493-4e0e-be3e-771d73140c45-kube-api-access-xz8xw\") pod \"payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj\" (UID: \"7d0d1284-2493-4e0e-be3e-771d73140c45\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" Sep 19 05:59:49.785103 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.785067 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:49.799939 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:49.799912 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" Sep 19 05:59:50.076286 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.076257 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj"] Sep 19 05:59:50.078939 ip-10-0-142-57 kubenswrapper[2584]: W0919 05:59:50.078906 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0d1284_2493_4e0e_be3e_771d73140c45.slice/crio-551b1257661e9a71a3689d72dd4e44e9ef8f924cbe8c9ac5f6eece8f5241b8ed WatchSource:0}: Error finding container 551b1257661e9a71a3689d72dd4e44e9ef8f924cbe8c9ac5f6eece8f5241b8ed: Status 404 returned error can't find the container with id 551b1257661e9a71a3689d72dd4e44e9ef8f924cbe8c9ac5f6eece8f5241b8ed Sep 19 05:59:50.380202 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.380167 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" event={"ID":"7d0d1284-2493-4e0e-be3e-771d73140c45","Type":"ContainerStarted","Data":"b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0"} Sep 19 05:59:50.380437 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.380227 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" event={"ID":"7d0d1284-2493-4e0e-be3e-771d73140c45","Type":"ContainerStarted","Data":"551b1257661e9a71a3689d72dd4e44e9ef8f924cbe8c9ac5f6eece8f5241b8ed"} Sep 19 05:59:50.380437 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.380382 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" Sep 19 05:59:50.403456 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.403387 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" podStartSLOduration=1.403370952 podStartE2EDuration="1.403370952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 05:59:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 05:59:50.40098614 +0000 UTC m=+1118.394213673" watchObservedRunningTime="2026-09-19 05:59:50.403370952 +0000 UTC m=+1118.396598483" Sep 19 05:59:50.410349 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.410315 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:50.438426 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.438375 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk"] Sep 19 05:59:50.438980 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.438902 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" podUID="e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55" containerName="payload-processing" containerID="cri-o://77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399" gracePeriod=30 Sep 19 05:59:50.447482 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.447382 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w"] Sep 19 05:59:50.447705 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.447677 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" podUID="6b6d8adb-bd30-41f5-98a5-9dce74f1404f" containerName="payload-pre-processing" containerID="cri-o://b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e" gracePeriod=30 Sep 19 05:59:50.482559 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.482527 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:50.581970 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.581943 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:50.646928 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.645864 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4"] Sep 19 05:59:50.646928 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.646718 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:50.649265 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.649237 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4b78e2-hj9k4"] Sep 19 05:59:50.698091 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.698065 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" Sep 19 05:59:50.758155 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.758129 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" Sep 19 05:59:50.836116 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.836079 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf6jg\" (UniqueName: \"kubernetes.io/projected/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55-kube-api-access-tf6jg\") pod \"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55\" (UID: \"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55\") " Sep 19 05:59:50.836318 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.836164 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55-plugins-config-volume\") pod \"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55\" (UID: \"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55\") " Sep 19 05:59:50.836318 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.836191 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b6d8adb-bd30-41f5-98a5-9dce74f1404f-plugins-config-volume\") pod \"6b6d8adb-bd30-41f5-98a5-9dce74f1404f\" (UID: \"6b6d8adb-bd30-41f5-98a5-9dce74f1404f\") " Sep 19 05:59:50.836318 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.836276 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdrtv\" (UniqueName: \"kubernetes.io/projected/6b6d8adb-bd30-41f5-98a5-9dce74f1404f-kube-api-access-hdrtv\") pod \"6b6d8adb-bd30-41f5-98a5-9dce74f1404f\" (UID: \"6b6d8adb-bd30-41f5-98a5-9dce74f1404f\") " Sep 19 05:59:50.836625 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.836599 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b6d8adb-bd30-41f5-98a5-9dce74f1404f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b6d8adb-bd30-41f5-98a5-9dce74f1404f" (UID: "6b6d8adb-bd30-41f5-98a5-9dce74f1404f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:59:50.836625 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.836604 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55" (UID: "e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 05:59:50.836801 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.836729 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:59:50.836801 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.836749 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b6d8adb-bd30-41f5-98a5-9dce74f1404f-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:59:50.839099 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.839072 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55-kube-api-access-tf6jg" (OuterVolumeSpecName: "kube-api-access-tf6jg") pod "e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55" (UID: "e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55"). InnerVolumeSpecName "kube-api-access-tf6jg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:59:50.839178 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.839113 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b6d8adb-bd30-41f5-98a5-9dce74f1404f-kube-api-access-hdrtv" (OuterVolumeSpecName: "kube-api-access-hdrtv") pod "6b6d8adb-bd30-41f5-98a5-9dce74f1404f" (UID: "6b6d8adb-bd30-41f5-98a5-9dce74f1404f"). InnerVolumeSpecName "kube-api-access-hdrtv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 05:59:50.872619 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.872594 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:50.938025 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.937945 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf6jg\" (UniqueName: \"kubernetes.io/projected/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55-kube-api-access-tf6jg\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:59:50.938025 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:50.937973 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdrtv\" (UniqueName: \"kubernetes.io/projected/6b6d8adb-bd30-41f5-98a5-9dce74f1404f-kube-api-access-hdrtv\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 05:59:51.014425 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.014370 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:51.387316 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.387275 2584 generic.go:356] "Generic (PLEG): container finished" podID="e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55" containerID="77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399" exitCode=0 Sep 19 05:59:51.387316 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.387306 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" event={"ID":"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55","Type":"ContainerDied","Data":"77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399"} Sep 19 05:59:51.387893 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.387326 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" Sep 19 05:59:51.387893 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.387359 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk" event={"ID":"e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55","Type":"ContainerDied","Data":"578101d2b40bf8ea96e1f225aff4aa5dd5efaa0c55cc39408f33d70cd4d13c82"} Sep 19 05:59:51.387893 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.387383 2584 scope.go:117] "RemoveContainer" containerID="77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399" Sep 19 05:59:51.388880 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.388861 2584 generic.go:356] "Generic (PLEG): container finished" podID="6b6d8adb-bd30-41f5-98a5-9dce74f1404f" containerID="b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e" exitCode=0 Sep 19 05:59:51.388948 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.388909 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" Sep 19 05:59:51.388948 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.388925 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" event={"ID":"6b6d8adb-bd30-41f5-98a5-9dce74f1404f","Type":"ContainerDied","Data":"b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e"} Sep 19 05:59:51.389034 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.388976 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w" event={"ID":"6b6d8adb-bd30-41f5-98a5-9dce74f1404f","Type":"ContainerDied","Data":"4e14f055e28bd01537611f08e8a1965e4b8d25d02db6a9772e6c44eb6c657911"} Sep 19 05:59:51.398838 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.398818 2584 scope.go:117] "RemoveContainer" containerID="77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399" Sep 19 05:59:51.399249 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:59:51.399154 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399\": container with ID starting with 77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399 not found: ID does not exist" containerID="77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399" Sep 19 05:59:51.399249 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.399184 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399"} err="failed to get container status \"77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399\": rpc error: code = NotFound desc = could not find container \"77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399\": container with ID starting with 77f4b8c6748715fb607dbc9739cbbdc63e2c70513bb397cd8ce1412d213c1399 not found: ID does not exist" Sep 19 05:59:51.399249 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.399207 2584 scope.go:117] "RemoveContainer" containerID="b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e" Sep 19 05:59:51.408097 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.408071 2584 scope.go:117] "RemoveContainer" containerID="b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e" Sep 19 05:59:51.408480 ip-10-0-142-57 kubenswrapper[2584]: E0919 05:59:51.408434 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e\": container with ID starting with b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e not found: ID does not exist" containerID="b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e" Sep 19 05:59:51.408586 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.408475 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e"} err="failed to get container status \"b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e\": rpc error: code = NotFound desc = could not find container \"b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e\": container with ID starting with b9db326aa7872e305afb5eeb974a1e4af808af683a21486047e3b7a4b7bc127e not found: ID does not exist" Sep 19 05:59:51.410466 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.410447 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk"] Sep 19 05:59:51.414959 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.414939 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4b78e2-866cff5565-lbqlk"] Sep 19 05:59:51.424260 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.424239 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w"] Sep 19 05:59:51.430840 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.430817 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4b78e2-7b5f77b54b-ptc6w"] Sep 19 05:59:51.737725 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:51.737630 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:52.268187 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:52.268153 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b6d8adb-bd30-41f5-98a5-9dce74f1404f" path="/var/lib/kubelet/pods/6b6d8adb-bd30-41f5-98a5-9dce74f1404f/volumes" Sep 19 05:59:52.268504 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:52.268491 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a245b83-853c-4c3f-af6a-d308cd02da06" path="/var/lib/kubelet/pods/9a245b83-853c-4c3f-af6a-d308cd02da06/volumes" Sep 19 05:59:52.268829 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:52.268817 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55" path="/var/lib/kubelet/pods/e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55/volumes" Sep 19 05:59:52.705461 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:52.705379 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:53.503438 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:53.503379 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:54.472952 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:54.472919 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:54.655802 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:54.655771 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:55.218861 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:55.218815 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:55.405363 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:55.405328 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:56.021099 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:56.021066 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 05:59:56.106193 ip-10-0-142-57 kubenswrapper[2584]: I0919 05:59:56.106163 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:00.141566 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.141523 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs"] Sep 19 06:00:00.142017 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.141999 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55" containerName="payload-processing" Sep 19 06:00:00.142096 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.142020 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55" containerName="payload-processing" Sep 19 06:00:00.142096 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.142045 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b6d8adb-bd30-41f5-98a5-9dce74f1404f" containerName="payload-pre-processing" Sep 19 06:00:00.142096 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.142051 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b6d8adb-bd30-41f5-98a5-9dce74f1404f" containerName="payload-pre-processing" Sep 19 06:00:00.142229 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.142150 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b6d8adb-bd30-41f5-98a5-9dce74f1404f" containerName="payload-pre-processing" Sep 19 06:00:00.142229 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.142166 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9aeaee4-5cb0-4f12-9ca8-2d5df19fff55" containerName="payload-processing" Sep 19 06:00:00.147155 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.147131 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs"] Sep 19 06:00:00.147314 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.147237 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs" Sep 19 06:00:00.150427 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.150298 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:00:00.157623 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.157281 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829960-fx5mq"] Sep 19 06:00:00.159250 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.159139 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs" Sep 19 06:00:00.163614 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.163495 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs" Sep 19 06:00:00.168322 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.166896 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6"] Sep 19 06:00:00.168322 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.167160 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829960-fx5mq" Sep 19 06:00:00.171542 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.171515 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs"] Sep 19 06:00:00.171671 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.171558 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs"] Sep 19 06:00:00.171671 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.171640 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6" Sep 19 06:00:00.172387 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.172363 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6"] Sep 19 06:00:00.181925 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.181900 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs" Sep 19 06:00:00.183656 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.183556 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829960-fx5mq"] Sep 19 06:00:00.227251 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.227133 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829960-fx5mq" Sep 19 06:00:00.241015 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.240796 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6" Sep 19 06:00:00.451703 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.451673 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs"] Sep 19 06:00:00.453081 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:00:00.453053 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0985253b_4c47_4993_ba51_583328402272.slice/crio-44187709079c793f31a4c5a183d6d53bccc90b86f03b990628719455ccc6a873 WatchSource:0}: Error finding container 44187709079c793f31a4c5a183d6d53bccc90b86f03b990628719455ccc6a873: Status 404 returned error can't find the container with id 44187709079c793f31a4c5a183d6d53bccc90b86f03b990628719455ccc6a873 Sep 19 06:00:00.474301 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.474250 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs"] Sep 19 06:00:00.476079 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:00:00.476051 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb304f2cf_92c2_4d7b_ae5c_2847d6fedba1.slice/crio-3f697dc0a968de7750832306319bde447af2844cf02e0f900f8e7a8879f3c031 WatchSource:0}: Error finding container 3f697dc0a968de7750832306319bde447af2844cf02e0f900f8e7a8879f3c031: Status 404 returned error can't find the container with id 3f697dc0a968de7750832306319bde447af2844cf02e0f900f8e7a8879f3c031 Sep 19 06:00:00.540097 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.540066 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829960-fx5mq"] Sep 19 06:00:00.541480 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:00:00.541452 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07bab225_a9be_4379_97e4_aa04e662c76a.slice/crio-47b64e0261b74ea65acfea2ae166b14e619ea3f2df82bf9e852f0430ec53727c WatchSource:0}: Error finding container 47b64e0261b74ea65acfea2ae166b14e619ea3f2df82bf9e852f0430ec53727c: Status 404 returned error can't find the container with id 47b64e0261b74ea65acfea2ae166b14e619ea3f2df82bf9e852f0430ec53727c Sep 19 06:00:00.542465 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:00.542439 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6"] Sep 19 06:00:00.543205 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:00:00.543184 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod856d5555_fb51_4f0b_b153_7d8d48a8b2a3.slice/crio-15f97870dfad0699ecc2b1571f86169b391925992a5362540a86de4f278d96d7 WatchSource:0}: Error finding container 15f97870dfad0699ecc2b1571f86169b391925992a5362540a86de4f278d96d7: Status 404 returned error can't find the container with id 15f97870dfad0699ecc2b1571f86169b391925992a5362540a86de4f278d96d7 Sep 19 06:00:01.391139 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.391110 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" Sep 19 06:00:01.452925 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.452882 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6" event={"ID":"856d5555-fb51-4f0b-b153-7d8d48a8b2a3","Type":"ContainerStarted","Data":"461b0a3f2f1b7b2205469c758aa27fe5860f795b1b95206bfa415230cbd00744"} Sep 19 06:00:01.453122 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.452945 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6" event={"ID":"856d5555-fb51-4f0b-b153-7d8d48a8b2a3","Type":"ContainerStarted","Data":"15f97870dfad0699ecc2b1571f86169b391925992a5362540a86de4f278d96d7"} Sep 19 06:00:01.454961 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.454930 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs" event={"ID":"b304f2cf-92c2-4d7b-ae5c-2847d6fedba1","Type":"ContainerStarted","Data":"078311a50e32d955f1edfe597914deee209fb48d2a84a59f93f63dee407ae71f"} Sep 19 06:00:01.455110 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.454981 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs" event={"ID":"b304f2cf-92c2-4d7b-ae5c-2847d6fedba1","Type":"ContainerStarted","Data":"3f697dc0a968de7750832306319bde447af2844cf02e0f900f8e7a8879f3c031"} Sep 19 06:00:01.456978 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.456867 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829960-fx5mq" event={"ID":"07bab225-a9be-4379-97e4-aa04e662c76a","Type":"ContainerStarted","Data":"d945f542fa3bee9e111461203c62124ceae2fcd9317ea3794a6902fdda83c4b2"} Sep 19 06:00:01.456978 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.456940 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829960-fx5mq" event={"ID":"07bab225-a9be-4379-97e4-aa04e662c76a","Type":"ContainerStarted","Data":"47b64e0261b74ea65acfea2ae166b14e619ea3f2df82bf9e852f0430ec53727c"} Sep 19 06:00:01.459202 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.459175 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs" event={"ID":"0985253b-4c47-4993-ba51-583328402272","Type":"ContainerStarted","Data":"dd5eaec97724cbc86c92f70dfb3a3420d2f792a36cad7f78422d685535bcc4d6"} Sep 19 06:00:01.459337 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.459219 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs" event={"ID":"0985253b-4c47-4993-ba51-583328402272","Type":"ContainerStarted","Data":"44187709079c793f31a4c5a183d6d53bccc90b86f03b990628719455ccc6a873"} Sep 19 06:00:01.469743 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.469692 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6" podStartSLOduration=1.469675492 podStartE2EDuration="1.469675492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:01.468594971 +0000 UTC m=+1129.461822502" watchObservedRunningTime="2026-09-19 06:00:01.469675492 +0000 UTC m=+1129.462903081" Sep 19 06:00:01.483923 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.483862 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs" podStartSLOduration=1.483848117 podStartE2EDuration="1.483848117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:01.482627095 +0000 UTC m=+1129.475854631" watchObservedRunningTime="2026-09-19 06:00:01.483848117 +0000 UTC m=+1129.477075706" Sep 19 06:00:01.499806 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.499715 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829960-fx5mq" podStartSLOduration=1.4996955490000001 podStartE2EDuration="1.499695549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:01.49720044 +0000 UTC m=+1129.490427972" watchObservedRunningTime="2026-09-19 06:00:01.499695549 +0000 UTC m=+1129.492923079" Sep 19 06:00:01.512795 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:01.512736 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs" podStartSLOduration=1.512714466 podStartE2EDuration="1.512714466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:01.51170362 +0000 UTC m=+1129.504931167" watchObservedRunningTime="2026-09-19 06:00:01.512714466 +0000 UTC m=+1129.505942000" Sep 19 06:00:02.467659 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:02.467620 2584 generic.go:356] "Generic (PLEG): container finished" podID="07bab225-a9be-4379-97e4-aa04e662c76a" containerID="d945f542fa3bee9e111461203c62124ceae2fcd9317ea3794a6902fdda83c4b2" exitCode=0 Sep 19 06:00:02.468135 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:02.467702 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829960-fx5mq" event={"ID":"07bab225-a9be-4379-97e4-aa04e662c76a","Type":"ContainerDied","Data":"d945f542fa3bee9e111461203c62124ceae2fcd9317ea3794a6902fdda83c4b2"} Sep 19 06:00:02.469684 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:02.469660 2584 generic.go:356] "Generic (PLEG): container finished" podID="0985253b-4c47-4993-ba51-583328402272" containerID="dd5eaec97724cbc86c92f70dfb3a3420d2f792a36cad7f78422d685535bcc4d6" exitCode=0 Sep 19 06:00:02.469804 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:02.469739 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs" event={"ID":"0985253b-4c47-4993-ba51-583328402272","Type":"ContainerDied","Data":"dd5eaec97724cbc86c92f70dfb3a3420d2f792a36cad7f78422d685535bcc4d6"} Sep 19 06:00:02.471594 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:02.471574 2584 generic.go:356] "Generic (PLEG): container finished" podID="856d5555-fb51-4f0b-b153-7d8d48a8b2a3" containerID="461b0a3f2f1b7b2205469c758aa27fe5860f795b1b95206bfa415230cbd00744" exitCode=0 Sep 19 06:00:02.471698 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:02.471657 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6" event={"ID":"856d5555-fb51-4f0b-b153-7d8d48a8b2a3","Type":"ContainerDied","Data":"461b0a3f2f1b7b2205469c758aa27fe5860f795b1b95206bfa415230cbd00744"} Sep 19 06:00:02.473335 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:02.473316 2584 generic.go:356] "Generic (PLEG): container finished" podID="b304f2cf-92c2-4d7b-ae5c-2847d6fedba1" containerID="078311a50e32d955f1edfe597914deee209fb48d2a84a59f93f63dee407ae71f" exitCode=0 Sep 19 06:00:02.473440 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:02.473352 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs" event={"ID":"b304f2cf-92c2-4d7b-ae5c-2847d6fedba1","Type":"ContainerDied","Data":"078311a50e32d955f1edfe597914deee209fb48d2a84a59f93f63dee407ae71f"} Sep 19 06:00:03.645507 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:03.645475 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6" Sep 19 06:00:03.711239 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:03.711214 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829960-fx5mq" Sep 19 06:00:03.714657 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:03.714633 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs" Sep 19 06:00:03.718268 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:03.718248 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs" Sep 19 06:00:04.487384 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:04.487348 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs" Sep 19 06:00:04.487384 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:04.487358 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs" event={"ID":"b304f2cf-92c2-4d7b-ae5c-2847d6fedba1","Type":"ContainerDied","Data":"3f697dc0a968de7750832306319bde447af2844cf02e0f900f8e7a8879f3c031"} Sep 19 06:00:04.487662 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:04.487437 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f697dc0a968de7750832306319bde447af2844cf02e0f900f8e7a8879f3c031" Sep 19 06:00:04.488848 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:04.488819 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829960-fx5mq" Sep 19 06:00:04.488985 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:04.488865 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829960-fx5mq" event={"ID":"07bab225-a9be-4379-97e4-aa04e662c76a","Type":"ContainerDied","Data":"47b64e0261b74ea65acfea2ae166b14e619ea3f2df82bf9e852f0430ec53727c"} Sep 19 06:00:04.488985 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:04.488917 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47b64e0261b74ea65acfea2ae166b14e619ea3f2df82bf9e852f0430ec53727c" Sep 19 06:00:04.490437 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:04.490392 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs" Sep 19 06:00:04.490569 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:04.490435 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs" event={"ID":"0985253b-4c47-4993-ba51-583328402272","Type":"ContainerDied","Data":"44187709079c793f31a4c5a183d6d53bccc90b86f03b990628719455ccc6a873"} Sep 19 06:00:04.490569 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:04.490476 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44187709079c793f31a4c5a183d6d53bccc90b86f03b990628719455ccc6a873" Sep 19 06:00:04.492030 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:04.492010 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6" event={"ID":"856d5555-fb51-4f0b-b153-7d8d48a8b2a3","Type":"ContainerDied","Data":"15f97870dfad0699ecc2b1571f86169b391925992a5362540a86de4f278d96d7"} Sep 19 06:00:04.492030 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:04.492024 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6" Sep 19 06:00:04.492199 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:04.492044 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15f97870dfad0699ecc2b1571f86169b391925992a5362540a86de4f278d96d7" Sep 19 06:00:12.865090 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:12.865055 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:13.530947 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:13.530909 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:14.109819 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.109785 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:14.339956 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.339917 2584 scope.go:117] "RemoveContainer" containerID="21b48826f3b22135a1273bd39fb110c7ddb5d860e58f659029bc326b10eaf09b" Sep 19 06:00:14.365569 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.365542 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:14.978660 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.978625 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8099-c2e767"] Sep 19 06:00:14.979125 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.979112 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b304f2cf-92c2-4d7b-ae5c-2847d6fedba1" containerName="cleanup" Sep 19 06:00:14.979186 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.979126 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b304f2cf-92c2-4d7b-ae5c-2847d6fedba1" containerName="cleanup" Sep 19 06:00:14.979186 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.979137 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07bab225-a9be-4379-97e4-aa04e662c76a" containerName="cleanup" Sep 19 06:00:14.979186 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.979143 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="07bab225-a9be-4379-97e4-aa04e662c76a" containerName="cleanup" Sep 19 06:00:14.979186 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.979152 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="856d5555-fb51-4f0b-b153-7d8d48a8b2a3" containerName="cleanup" Sep 19 06:00:14.979186 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.979157 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="856d5555-fb51-4f0b-b153-7d8d48a8b2a3" containerName="cleanup" Sep 19 06:00:14.979186 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.979180 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0985253b-4c47-4993-ba51-583328402272" containerName="cleanup" Sep 19 06:00:14.979186 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.979185 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0985253b-4c47-4993-ba51-583328402272" containerName="cleanup" Sep 19 06:00:14.979423 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.979250 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="07bab225-a9be-4379-97e4-aa04e662c76a" containerName="cleanup" Sep 19 06:00:14.979423 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.979259 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="856d5555-fb51-4f0b-b153-7d8d48a8b2a3" containerName="cleanup" Sep 19 06:00:14.979423 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.979268 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b304f2cf-92c2-4d7b-ae5c-2847d6fedba1" containerName="cleanup" Sep 19 06:00:14.979423 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.979276 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0985253b-4c47-4993-ba51-583328402272" containerName="cleanup" Sep 19 06:00:14.983728 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.983710 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-c2e767" Sep 19 06:00:14.986360 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.986332 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tpl82\"" Sep 19 06:00:14.988007 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:14.987984 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8099-c2e767"] Sep 19 06:00:15.087707 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:15.087674 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz8z6\" (UniqueName: \"kubernetes.io/projected/23a583dd-b140-49c7-a926-1697ff1660c5-kube-api-access-gz8z6\") pod \"test-curl-8099-c2e767\" (UID: \"23a583dd-b140-49c7-a926-1697ff1660c5\") " pod="opendatahub/test-curl-8099-c2e767" Sep 19 06:00:15.188555 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:15.188513 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz8z6\" (UniqueName: \"kubernetes.io/projected/23a583dd-b140-49c7-a926-1697ff1660c5-kube-api-access-gz8z6\") pod \"test-curl-8099-c2e767\" (UID: \"23a583dd-b140-49c7-a926-1697ff1660c5\") " pod="opendatahub/test-curl-8099-c2e767" Sep 19 06:00:15.200498 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:15.200458 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz8z6\" (UniqueName: \"kubernetes.io/projected/23a583dd-b140-49c7-a926-1697ff1660c5-kube-api-access-gz8z6\") pod \"test-curl-8099-c2e767\" (UID: \"23a583dd-b140-49c7-a926-1697ff1660c5\") " pod="opendatahub/test-curl-8099-c2e767" Sep 19 06:00:15.206971 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:15.206942 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:15.294927 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:15.294841 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-c2e767" Sep 19 06:00:15.569432 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:15.569387 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8099-c2e767"] Sep 19 06:00:15.571001 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:00:15.570961 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23a583dd_b140_49c7_a926_1697ff1660c5.slice/crio-25c3ae0c77cc9a499a16739ffa5f07794663d63d12129150f97cd1ebe175458d WatchSource:0}: Error finding container 25c3ae0c77cc9a499a16739ffa5f07794663d63d12129150f97cd1ebe175458d: Status 404 returned error can't find the container with id 25c3ae0c77cc9a499a16739ffa5f07794663d63d12129150f97cd1ebe175458d Sep 19 06:00:15.573234 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:15.573215 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 06:00:16.105153 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:16.105121 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:16.570549 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:16.570448 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8099-c2e767" event={"ID":"23a583dd-b140-49c7-a926-1697ff1660c5","Type":"ContainerStarted","Data":"25c3ae0c77cc9a499a16739ffa5f07794663d63d12129150f97cd1ebe175458d"} Sep 19 06:00:17.579141 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:17.579102 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8099-c2e767" event={"ID":"23a583dd-b140-49c7-a926-1697ff1660c5","Type":"ContainerStarted","Data":"d18d05e46b65076777a4cc79b9bdff3340cdc7cf239b33abbe2dc199010f5af0"} Sep 19 06:00:17.617287 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:17.617228 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8099-c2e767" podStartSLOduration=1.7496391839999998 podStartE2EDuration="3.617212105s" totalImagesPullingTime="1.867572921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:17.614652441 +0000 UTC m=+1145.607879972" watchObservedRunningTime="2026-09-19 06:00:17.617212105 +0000 UTC m=+1145.610439663" Sep 19 06:00:17.634799 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:17.634764 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:18.248783 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.248745 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8099-c2e767"] Sep 19 06:00:18.253575 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.253545 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8099-c2e767"] Sep 19 06:00:18.395022 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.394982 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8099-a72daf"] Sep 19 06:00:18.395505 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.395491 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23a583dd-b140-49c7-a926-1697ff1660c5" containerName="test-curl-8099-c2e767" Sep 19 06:00:18.395552 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.395506 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="23a583dd-b140-49c7-a926-1697ff1660c5" containerName="test-curl-8099-c2e767" Sep 19 06:00:18.395592 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.395583 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="23a583dd-b140-49c7-a926-1697ff1660c5" containerName="test-curl-8099-c2e767" Sep 19 06:00:18.398677 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.398662 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-a72daf" Sep 19 06:00:18.408163 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.408133 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8099-a72daf"] Sep 19 06:00:18.420226 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.420199 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htcwj\" (UniqueName: \"kubernetes.io/projected/25db16e0-32d2-4d63-b454-7b737dfd0594-kube-api-access-htcwj\") pod \"test-curl-8099-a72daf\" (UID: \"25db16e0-32d2-4d63-b454-7b737dfd0594\") " pod="opendatahub/test-curl-8099-a72daf" Sep 19 06:00:18.520757 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.520670 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htcwj\" (UniqueName: \"kubernetes.io/projected/25db16e0-32d2-4d63-b454-7b737dfd0594-kube-api-access-htcwj\") pod \"test-curl-8099-a72daf\" (UID: \"25db16e0-32d2-4d63-b454-7b737dfd0594\") " pod="opendatahub/test-curl-8099-a72daf" Sep 19 06:00:18.528841 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.528808 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htcwj\" (UniqueName: \"kubernetes.io/projected/25db16e0-32d2-4d63-b454-7b737dfd0594-kube-api-access-htcwj\") pod \"test-curl-8099-a72daf\" (UID: \"25db16e0-32d2-4d63-b454-7b737dfd0594\") " pod="opendatahub/test-curl-8099-a72daf" Sep 19 06:00:18.585792 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:00:18.585763 2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8099-c2e767" container="test-curl-8099-c2e767" Sep 19 06:00:18.708889 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.708856 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-a72daf" Sep 19 06:00:18.983262 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:18.983229 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8099-a72daf"] Sep 19 06:00:18.984707 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:00:18.984672 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25db16e0_32d2_4d63_b454_7b737dfd0594.slice/crio-0e76014d5eca0d89748d366b80a84bf302967fd2a88362c64bcf7d55f1cbab7a WatchSource:0}: Error finding container 0e76014d5eca0d89748d366b80a84bf302967fd2a88362c64bcf7d55f1cbab7a: Status 404 returned error can't find the container with id 0e76014d5eca0d89748d366b80a84bf302967fd2a88362c64bcf7d55f1cbab7a Sep 19 06:00:19.592919 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:19.592870 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8099-a72daf" event={"ID":"25db16e0-32d2-4d63-b454-7b737dfd0594","Type":"ContainerStarted","Data":"0fea5992dbef7d166ef4757550a56828eea8315566f4856461f86e5094b5f149"} Sep 19 06:00:19.593468 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:19.592929 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8099-a72daf" event={"ID":"25db16e0-32d2-4d63-b454-7b737dfd0594","Type":"ContainerStarted","Data":"0e76014d5eca0d89748d366b80a84bf302967fd2a88362c64bcf7d55f1cbab7a"} Sep 19 06:00:19.593468 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:19.593058 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8099-c2e767" podUID="23a583dd-b140-49c7-a926-1697ff1660c5" containerName="test-curl-8099-c2e767" containerID="cri-o://d18d05e46b65076777a4cc79b9bdff3340cdc7cf239b33abbe2dc199010f5af0" gracePeriod=2 Sep 19 06:00:19.612866 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:19.612813 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8099-a72daf" podStartSLOduration=1.612796887 podStartE2EDuration="1.612796887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:19.611531808 +0000 UTC m=+1147.604759339" watchObservedRunningTime="2026-09-19 06:00:19.612796887 +0000 UTC m=+1147.606024417" Sep 19 06:00:20.230261 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.230221 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8099-a72daf"] Sep 19 06:00:20.234673 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.234637 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8099-a72daf"] Sep 19 06:00:20.349910 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.349872 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8099-63cb33"] Sep 19 06:00:20.350480 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.350459 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25db16e0-32d2-4d63-b454-7b737dfd0594" containerName="test-curl-8099-a72daf" Sep 19 06:00:20.350591 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.350483 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="25db16e0-32d2-4d63-b454-7b737dfd0594" containerName="test-curl-8099-a72daf" Sep 19 06:00:20.350650 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.350594 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="25db16e0-32d2-4d63-b454-7b737dfd0594" containerName="test-curl-8099-a72daf" Sep 19 06:00:20.353645 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.353625 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-63cb33" Sep 19 06:00:20.360910 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.360890 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8099-63cb33"] Sep 19 06:00:20.440319 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.440284 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hhrv\" (UniqueName: \"kubernetes.io/projected/17d20b41-7d83-4343-a2dd-ac6a153eaac9-kube-api-access-9hhrv\") pod \"test-curl-8099-63cb33\" (UID: \"17d20b41-7d83-4343-a2dd-ac6a153eaac9\") " pod="opendatahub/test-curl-8099-63cb33" Sep 19 06:00:20.541392 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.541353 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hhrv\" (UniqueName: \"kubernetes.io/projected/17d20b41-7d83-4343-a2dd-ac6a153eaac9-kube-api-access-9hhrv\") pod \"test-curl-8099-63cb33\" (UID: \"17d20b41-7d83-4343-a2dd-ac6a153eaac9\") " pod="opendatahub/test-curl-8099-63cb33" Sep 19 06:00:20.548957 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.548928 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hhrv\" (UniqueName: \"kubernetes.io/projected/17d20b41-7d83-4343-a2dd-ac6a153eaac9-kube-api-access-9hhrv\") pod \"test-curl-8099-63cb33\" (UID: \"17d20b41-7d83-4343-a2dd-ac6a153eaac9\") " pod="opendatahub/test-curl-8099-63cb33" Sep 19 06:00:20.599435 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:00:20.599395 2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8099-a72daf" container="test-curl-8099-a72daf" Sep 19 06:00:20.664286 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.664196 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-63cb33" Sep 19 06:00:20.947465 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:20.947383 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8099-63cb33"] Sep 19 06:00:20.949761 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:00:20.949727 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17d20b41_7d83_4343_a2dd_ac6a153eaac9.slice/crio-37e633eaf99d0085c0a99a4b4fc33358dd037e73c6f3512159d960f35a77a784 WatchSource:0}: Error finding container 37e633eaf99d0085c0a99a4b4fc33358dd037e73c6f3512159d960f35a77a784: Status 404 returned error can't find the container with id 37e633eaf99d0085c0a99a4b4fc33358dd037e73c6f3512159d960f35a77a784 Sep 19 06:00:21.610631 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:21.610593 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8099-63cb33" event={"ID":"17d20b41-7d83-4343-a2dd-ac6a153eaac9","Type":"ContainerStarted","Data":"fa14c7704abbd7d72567310a52179191644285151718d08bfc9bfef5c9805487"} Sep 19 06:00:21.610998 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:21.610644 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8099-63cb33" event={"ID":"17d20b41-7d83-4343-a2dd-ac6a153eaac9","Type":"ContainerStarted","Data":"37e633eaf99d0085c0a99a4b4fc33358dd037e73c6f3512159d960f35a77a784"} Sep 19 06:00:21.610998 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:21.610719 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8099-a72daf" podUID="25db16e0-32d2-4d63-b454-7b737dfd0594" containerName="test-curl-8099-a72daf" containerID="cri-o://0fea5992dbef7d166ef4757550a56828eea8315566f4856461f86e5094b5f149" gracePeriod=2 Sep 19 06:00:21.627941 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:21.627888 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8099-63cb33" podStartSLOduration=1.627872867 podStartE2EDuration="1.627872867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:21.625708825 +0000 UTC m=+1149.618936359" watchObservedRunningTime="2026-09-19 06:00:21.627872867 +0000 UTC m=+1149.621100398" Sep 19 06:00:21.794032 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:21.794006 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-c2e767" Sep 19 06:00:21.855447 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:21.855351 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gz8z6\" (UniqueName: \"kubernetes.io/projected/23a583dd-b140-49c7-a926-1697ff1660c5-kube-api-access-gz8z6\") pod \"23a583dd-b140-49c7-a926-1697ff1660c5\" (UID: \"23a583dd-b140-49c7-a926-1697ff1660c5\") " Sep 19 06:00:21.857263 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:21.857242 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23a583dd-b140-49c7-a926-1697ff1660c5-kube-api-access-gz8z6" (OuterVolumeSpecName: "kube-api-access-gz8z6") pod "23a583dd-b140-49c7-a926-1697ff1660c5" (UID: "23a583dd-b140-49c7-a926-1697ff1660c5"). InnerVolumeSpecName "kube-api-access-gz8z6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:00:21.956953 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:21.956913 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz8z6\" (UniqueName: \"kubernetes.io/projected/23a583dd-b140-49c7-a926-1697ff1660c5-kube-api-access-gz8z6\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:00:22.235390 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.235298 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8099-63cb33"] Sep 19 06:00:22.239863 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.239832 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8099-63cb33"] Sep 19 06:00:22.267943 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.267910 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23a583dd-b140-49c7-a926-1697ff1660c5" path="/var/lib/kubelet/pods/23a583dd-b140-49c7-a926-1697ff1660c5/volumes" Sep 19 06:00:22.362216 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.362187 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8099-f5c784"] Sep 19 06:00:22.362774 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.362756 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17d20b41-7d83-4343-a2dd-ac6a153eaac9" containerName="test-curl-8099-63cb33" Sep 19 06:00:22.362890 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.362777 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="17d20b41-7d83-4343-a2dd-ac6a153eaac9" containerName="test-curl-8099-63cb33" Sep 19 06:00:22.362947 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.362915 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="17d20b41-7d83-4343-a2dd-ac6a153eaac9" containerName="test-curl-8099-63cb33" Sep 19 06:00:22.366317 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.366298 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-f5c784" Sep 19 06:00:22.375278 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.375255 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8099-f5c784"] Sep 19 06:00:22.462979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.462941 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2597\" (UniqueName: \"kubernetes.io/projected/fdce82b5-f1a5-4ace-b893-1b95c2e8a52f-kube-api-access-l2597\") pod \"test-curl-8099-f5c784\" (UID: \"fdce82b5-f1a5-4ace-b893-1b95c2e8a52f\") " pod="opendatahub/test-curl-8099-f5c784" Sep 19 06:00:22.563670 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.563586 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2597\" (UniqueName: \"kubernetes.io/projected/fdce82b5-f1a5-4ace-b893-1b95c2e8a52f-kube-api-access-l2597\") pod \"test-curl-8099-f5c784\" (UID: \"fdce82b5-f1a5-4ace-b893-1b95c2e8a52f\") " pod="opendatahub/test-curl-8099-f5c784" Sep 19 06:00:22.572118 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.572091 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2597\" (UniqueName: \"kubernetes.io/projected/fdce82b5-f1a5-4ace-b893-1b95c2e8a52f-kube-api-access-l2597\") pod \"test-curl-8099-f5c784\" (UID: \"fdce82b5-f1a5-4ace-b893-1b95c2e8a52f\") " pod="opendatahub/test-curl-8099-f5c784" Sep 19 06:00:22.619004 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.618972 2584 generic.go:356] "Generic (PLEG): container finished" podID="23a583dd-b140-49c7-a926-1697ff1660c5" containerID="d18d05e46b65076777a4cc79b9bdff3340cdc7cf239b33abbe2dc199010f5af0" exitCode=137 Sep 19 06:00:22.619501 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.619030 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-c2e767" Sep 19 06:00:22.619501 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.619097 2584 scope.go:117] "RemoveContainer" containerID="d18d05e46b65076777a4cc79b9bdff3340cdc7cf239b33abbe2dc199010f5af0" Sep 19 06:00:22.619501 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:00:22.619236 2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8099-63cb33" container="test-curl-8099-63cb33" Sep 19 06:00:22.628983 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.628966 2584 scope.go:117] "RemoveContainer" containerID="d18d05e46b65076777a4cc79b9bdff3340cdc7cf239b33abbe2dc199010f5af0" Sep 19 06:00:22.629287 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:00:22.629263 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d18d05e46b65076777a4cc79b9bdff3340cdc7cf239b33abbe2dc199010f5af0\": container with ID starting with d18d05e46b65076777a4cc79b9bdff3340cdc7cf239b33abbe2dc199010f5af0 not found: ID does not exist" containerID="d18d05e46b65076777a4cc79b9bdff3340cdc7cf239b33abbe2dc199010f5af0" Sep 19 06:00:22.629391 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.629291 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d18d05e46b65076777a4cc79b9bdff3340cdc7cf239b33abbe2dc199010f5af0"} err="failed to get container status \"d18d05e46b65076777a4cc79b9bdff3340cdc7cf239b33abbe2dc199010f5af0\": rpc error: code = NotFound desc = could not find container \"d18d05e46b65076777a4cc79b9bdff3340cdc7cf239b33abbe2dc199010f5af0\": container with ID starting with d18d05e46b65076777a4cc79b9bdff3340cdc7cf239b33abbe2dc199010f5af0 not found: ID does not exist" Sep 19 06:00:22.677265 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.677227 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-f5c784" Sep 19 06:00:22.957125 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:22.957098 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8099-f5c784"] Sep 19 06:00:22.958467 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:00:22.958436 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdce82b5_f1a5_4ace_b893_1b95c2e8a52f.slice/crio-dc5f460a6bb2a9546057e36f04fe51c400cfb6bcdfbf1e11df6c7ca74b09a977 WatchSource:0}: Error finding container dc5f460a6bb2a9546057e36f04fe51c400cfb6bcdfbf1e11df6c7ca74b09a977: Status 404 returned error can't find the container with id dc5f460a6bb2a9546057e36f04fe51c400cfb6bcdfbf1e11df6c7ca74b09a977 Sep 19 06:00:23.628753 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:23.628722 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8099-f5c784" event={"ID":"fdce82b5-f1a5-4ace-b893-1b95c2e8a52f","Type":"ContainerStarted","Data":"73f917c5478730852a87253a39594c78b7e00f901b000e2d98c93590d96133fd"} Sep 19 06:00:23.629111 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:23.628769 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8099-f5c784" event={"ID":"fdce82b5-f1a5-4ace-b893-1b95c2e8a52f","Type":"ContainerStarted","Data":"dc5f460a6bb2a9546057e36f04fe51c400cfb6bcdfbf1e11df6c7ca74b09a977"} Sep 19 06:00:23.629111 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:23.628785 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8099-63cb33" podUID="17d20b41-7d83-4343-a2dd-ac6a153eaac9" containerName="test-curl-8099-63cb33" containerID="cri-o://fa14c7704abbd7d72567310a52179191644285151718d08bfc9bfef5c9805487" gracePeriod=2 Sep 19 06:00:23.653904 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:23.653857 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8099-f5c784" podStartSLOduration=1.65384005 podStartE2EDuration="1.65384005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:23.651133142 +0000 UTC m=+1151.644360673" watchObservedRunningTime="2026-09-19 06:00:23.65384005 +0000 UTC m=+1151.647067611" Sep 19 06:00:23.794394 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:23.794367 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-a72daf" Sep 19 06:00:23.878656 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:23.878583 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htcwj\" (UniqueName: \"kubernetes.io/projected/25db16e0-32d2-4d63-b454-7b737dfd0594-kube-api-access-htcwj\") pod \"25db16e0-32d2-4d63-b454-7b737dfd0594\" (UID: \"25db16e0-32d2-4d63-b454-7b737dfd0594\") " Sep 19 06:00:23.880499 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:23.880461 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25db16e0-32d2-4d63-b454-7b737dfd0594-kube-api-access-htcwj" (OuterVolumeSpecName: "kube-api-access-htcwj") pod "25db16e0-32d2-4d63-b454-7b737dfd0594" (UID: "25db16e0-32d2-4d63-b454-7b737dfd0594"). InnerVolumeSpecName "kube-api-access-htcwj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:00:23.980616 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:23.980582 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htcwj\" (UniqueName: \"kubernetes.io/projected/25db16e0-32d2-4d63-b454-7b737dfd0594-kube-api-access-htcwj\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:00:24.250922 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:24.250837 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8099-f5c784"] Sep 19 06:00:24.254942 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:24.254911 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8099-f5c784"] Sep 19 06:00:24.278459 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:24.278425 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25db16e0-32d2-4d63-b454-7b737dfd0594" path="/var/lib/kubelet/pods/25db16e0-32d2-4d63-b454-7b737dfd0594/volumes" Sep 19 06:00:24.636165 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:24.636134 2584 generic.go:356] "Generic (PLEG): container finished" podID="25db16e0-32d2-4d63-b454-7b737dfd0594" containerID="0fea5992dbef7d166ef4757550a56828eea8315566f4856461f86e5094b5f149" exitCode=137 Sep 19 06:00:24.636620 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:24.636203 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-a72daf" Sep 19 06:00:24.636620 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:24.636237 2584 scope.go:117] "RemoveContainer" containerID="0fea5992dbef7d166ef4757550a56828eea8315566f4856461f86e5094b5f149" Sep 19 06:00:24.636620 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:00:24.636449 2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8099-f5c784" container="test-curl-8099-f5c784" Sep 19 06:00:24.645607 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:24.645589 2584 scope.go:117] "RemoveContainer" containerID="0fea5992dbef7d166ef4757550a56828eea8315566f4856461f86e5094b5f149" Sep 19 06:00:24.645881 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:00:24.645863 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fea5992dbef7d166ef4757550a56828eea8315566f4856461f86e5094b5f149\": container with ID starting with 0fea5992dbef7d166ef4757550a56828eea8315566f4856461f86e5094b5f149 not found: ID does not exist" containerID="0fea5992dbef7d166ef4757550a56828eea8315566f4856461f86e5094b5f149" Sep 19 06:00:24.645940 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:24.645894 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fea5992dbef7d166ef4757550a56828eea8315566f4856461f86e5094b5f149"} err="failed to get container status \"0fea5992dbef7d166ef4757550a56828eea8315566f4856461f86e5094b5f149\": rpc error: code = NotFound desc = could not find container \"0fea5992dbef7d166ef4757550a56828eea8315566f4856461f86e5094b5f149\": container with ID starting with 0fea5992dbef7d166ef4757550a56828eea8315566f4856461f86e5094b5f149 not found: ID does not exist" Sep 19 06:00:24.888647 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:24.888567 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:25.195903 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.195776 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq"] Sep 19 06:00:25.196320 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.196301 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdce82b5-f1a5-4ace-b893-1b95c2e8a52f" containerName="test-curl-8099-f5c784" Sep 19 06:00:25.196320 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.196320 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdce82b5-f1a5-4ace-b893-1b95c2e8a52f" containerName="test-curl-8099-f5c784" Sep 19 06:00:25.196509 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.196413 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdce82b5-f1a5-4ace-b893-1b95c2e8a52f" containerName="test-curl-8099-f5c784" Sep 19 06:00:25.200587 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.200570 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.203243 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.203222 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-957d1190-openshift-default-dockercfg-5xqjl\"" Sep 19 06:00:25.213853 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.213830 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq"] Sep 19 06:00:25.229539 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.229498 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:25.292838 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.292803 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-envoy\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.293018 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.292848 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-workload-certs\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.293018 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.292908 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2dr2\" (UniqueName: \"kubernetes.io/projected/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-kube-api-access-v2dr2\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.293018 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.292941 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-credential-socket\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.293018 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.292973 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istiod-ca-cert\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.293018 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.293000 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-token\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.293189 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.293061 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-workload-socket\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.293189 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.293145 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-podinfo\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.293254 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.293190 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-data\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.328803 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.328768 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:25.394280 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.394241 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-podinfo\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.394516 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.394292 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-data\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.394516 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.394334 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-envoy\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.394516 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.394368 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-workload-certs\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.394516 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.394447 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2dr2\" (UniqueName: \"kubernetes.io/projected/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-kube-api-access-v2dr2\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.394516 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.394480 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-credential-socket\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.394800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.394520 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istiod-ca-cert\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.394800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.394561 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-token\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.394800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.394597 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-workload-socket\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.394949 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.394834 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-data\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.394949 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.394847 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-workload-certs\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.394949 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.394921 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-workload-socket\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.395112 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.395079 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-credential-socket\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.395546 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.395522 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istiod-ca-cert\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.396761 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.396734 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-envoy\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.396897 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.396819 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-podinfo\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.402102 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.402078 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-token\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.402594 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.402568 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2dr2\" (UniqueName: \"kubernetes.io/projected/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-kube-api-access-v2dr2\") pod \"e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.510806 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.510715 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:25.643977 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.643943 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8099-f5c784" podUID="fdce82b5-f1a5-4ace-b893-1b95c2e8a52f" containerName="test-curl-8099-f5c784" containerID="cri-o://73f917c5478730852a87253a39594c78b7e00f901b000e2d98c93590d96133fd" gracePeriod=2 Sep 19 06:00:25.791524 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.791495 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq"] Sep 19 06:00:25.792354 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:00:25.792323 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21dcd431_5ee8_4a82_bf15_71e9fd9b79a3.slice/crio-54d0ff9279c90c56cebb6b82869fe4957c191b85fa914974ba89e583d5235a96 WatchSource:0}: Error finding container 54d0ff9279c90c56cebb6b82869fe4957c191b85fa914974ba89e583d5235a96: Status 404 returned error can't find the container with id 54d0ff9279c90c56cebb6b82869fe4957c191b85fa914974ba89e583d5235a96 Sep 19 06:00:25.794765 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.794726 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:00:25.794835 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.794800 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:00:25.794882 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.794834 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:00:25.796044 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.796022 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-63cb33" Sep 19 06:00:25.901298 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.901259 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hhrv\" (UniqueName: \"kubernetes.io/projected/17d20b41-7d83-4343-a2dd-ac6a153eaac9-kube-api-access-9hhrv\") pod \"17d20b41-7d83-4343-a2dd-ac6a153eaac9\" (UID: \"17d20b41-7d83-4343-a2dd-ac6a153eaac9\") " Sep 19 06:00:25.903267 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:25.903244 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17d20b41-7d83-4343-a2dd-ac6a153eaac9-kube-api-access-9hhrv" (OuterVolumeSpecName: "kube-api-access-9hhrv") pod "17d20b41-7d83-4343-a2dd-ac6a153eaac9" (UID: "17d20b41-7d83-4343-a2dd-ac6a153eaac9"). InnerVolumeSpecName "kube-api-access-9hhrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:00:26.003222 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:26.003183 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hhrv\" (UniqueName: \"kubernetes.io/projected/17d20b41-7d83-4343-a2dd-ac6a153eaac9-kube-api-access-9hhrv\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:00:26.268619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:26.268587 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17d20b41-7d83-4343-a2dd-ac6a153eaac9" path="/var/lib/kubelet/pods/17d20b41-7d83-4343-a2dd-ac6a153eaac9/volumes" Sep 19 06:00:26.652358 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:26.652326 2584 generic.go:356] "Generic (PLEG): container finished" podID="17d20b41-7d83-4343-a2dd-ac6a153eaac9" containerID="fa14c7704abbd7d72567310a52179191644285151718d08bfc9bfef5c9805487" exitCode=137 Sep 19 06:00:26.652815 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:26.652374 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-63cb33" Sep 19 06:00:26.652815 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:26.652450 2584 scope.go:117] "RemoveContainer" containerID="fa14c7704abbd7d72567310a52179191644285151718d08bfc9bfef5c9805487" Sep 19 06:00:26.654477 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:26.654441 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" event={"ID":"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3","Type":"ContainerStarted","Data":"3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb"} Sep 19 06:00:26.654621 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:26.654509 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" event={"ID":"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3","Type":"ContainerStarted","Data":"54d0ff9279c90c56cebb6b82869fe4957c191b85fa914974ba89e583d5235a96"} Sep 19 06:00:26.662175 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:26.662159 2584 scope.go:117] "RemoveContainer" containerID="fa14c7704abbd7d72567310a52179191644285151718d08bfc9bfef5c9805487" Sep 19 06:00:26.662467 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:00:26.662450 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa14c7704abbd7d72567310a52179191644285151718d08bfc9bfef5c9805487\": container with ID starting with fa14c7704abbd7d72567310a52179191644285151718d08bfc9bfef5c9805487 not found: ID does not exist" containerID="fa14c7704abbd7d72567310a52179191644285151718d08bfc9bfef5c9805487" Sep 19 06:00:26.662524 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:26.662475 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa14c7704abbd7d72567310a52179191644285151718d08bfc9bfef5c9805487"} err="failed to get container status \"fa14c7704abbd7d72567310a52179191644285151718d08bfc9bfef5c9805487\": rpc error: code = NotFound desc = could not find container \"fa14c7704abbd7d72567310a52179191644285151718d08bfc9bfef5c9805487\": container with ID starting with fa14c7704abbd7d72567310a52179191644285151718d08bfc9bfef5c9805487 not found: ID does not exist" Sep 19 06:00:26.675964 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:26.675909 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" podStartSLOduration=1.675892008 podStartE2EDuration="1.675892008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:26.675278771 +0000 UTC m=+1154.668506305" watchObservedRunningTime="2026-09-19 06:00:26.675892008 +0000 UTC m=+1154.669119539" Sep 19 06:00:27.511112 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:27.511075 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:27.516482 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:27.516453 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:27.661835 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:27.661810 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:27.663230 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:27.663206 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:27.850486 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:27.850462 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-f5c784" Sep 19 06:00:27.923325 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:27.923295 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2597\" (UniqueName: \"kubernetes.io/projected/fdce82b5-f1a5-4ace-b893-1b95c2e8a52f-kube-api-access-l2597\") pod \"fdce82b5-f1a5-4ace-b893-1b95c2e8a52f\" (UID: \"fdce82b5-f1a5-4ace-b893-1b95c2e8a52f\") " Sep 19 06:00:27.925410 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:27.925372 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdce82b5-f1a5-4ace-b893-1b95c2e8a52f-kube-api-access-l2597" (OuterVolumeSpecName: "kube-api-access-l2597") pod "fdce82b5-f1a5-4ace-b893-1b95c2e8a52f" (UID: "fdce82b5-f1a5-4ace-b893-1b95c2e8a52f"). InnerVolumeSpecName "kube-api-access-l2597". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:00:28.024756 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:28.024668 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2597\" (UniqueName: \"kubernetes.io/projected/fdce82b5-f1a5-4ace-b893-1b95c2e8a52f-kube-api-access-l2597\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:00:28.275818 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:28.275728 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdce82b5-f1a5-4ace-b893-1b95c2e8a52f" path="/var/lib/kubelet/pods/fdce82b5-f1a5-4ace-b893-1b95c2e8a52f/volumes" Sep 19 06:00:28.567834 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:28.567749 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:28.657329 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:28.657294 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:28.670028 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:28.669991 2584 generic.go:356] "Generic (PLEG): container finished" podID="fdce82b5-f1a5-4ace-b893-1b95c2e8a52f" containerID="73f917c5478730852a87253a39594c78b7e00f901b000e2d98c93590d96133fd" exitCode=137 Sep 19 06:00:28.670470 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:28.670045 2584 scope.go:117] "RemoveContainer" containerID="73f917c5478730852a87253a39594c78b7e00f901b000e2d98c93590d96133fd" Sep 19 06:00:28.670470 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:28.670063 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8099-f5c784" Sep 19 06:00:28.679566 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:28.679545 2584 scope.go:117] "RemoveContainer" containerID="73f917c5478730852a87253a39594c78b7e00f901b000e2d98c93590d96133fd" Sep 19 06:00:28.679891 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:00:28.679870 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73f917c5478730852a87253a39594c78b7e00f901b000e2d98c93590d96133fd\": container with ID starting with 73f917c5478730852a87253a39594c78b7e00f901b000e2d98c93590d96133fd not found: ID does not exist" containerID="73f917c5478730852a87253a39594c78b7e00f901b000e2d98c93590d96133fd" Sep 19 06:00:28.679950 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:28.679902 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73f917c5478730852a87253a39594c78b7e00f901b000e2d98c93590d96133fd"} err="failed to get container status \"73f917c5478730852a87253a39594c78b7e00f901b000e2d98c93590d96133fd\": rpc error: code = NotFound desc = could not find container \"73f917c5478730852a87253a39594c78b7e00f901b000e2d98c93590d96133fd\": container with ID starting with 73f917c5478730852a87253a39594c78b7e00f901b000e2d98c93590d96133fd not found: ID does not exist" Sep 19 06:00:29.586000 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:29.585970 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:30.395639 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:30.395602 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:31.115600 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:31.115568 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:31.861293 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:31.861259 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:32.944786 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:32.944750 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:33.065200 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.065166 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:33.237209 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.237120 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:33.331165 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.331126 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:33.333033 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.333011 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb"] Sep 19 06:00:33.336731 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.336715 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" Sep 19 06:00:33.339082 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.339064 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-957d1190-dockercfg-xcfhv\"" Sep 19 06:00:33.339177 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.339117 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-957d1190\"" Sep 19 06:00:33.347624 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.347599 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb"] Sep 19 06:00:33.480208 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.480164 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c90814f7-f992-4a33-bc3a-136dd2265398-plugins-config-volume\") pod \"payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb\" (UID: \"c90814f7-f992-4a33-bc3a-136dd2265398\") " pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" Sep 19 06:00:33.480414 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.480218 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x87sr\" (UniqueName: \"kubernetes.io/projected/c90814f7-f992-4a33-bc3a-136dd2265398-kube-api-access-x87sr\") pod \"payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb\" (UID: \"c90814f7-f992-4a33-bc3a-136dd2265398\") " pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" Sep 19 06:00:33.580874 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.580842 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c90814f7-f992-4a33-bc3a-136dd2265398-plugins-config-volume\") pod \"payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb\" (UID: \"c90814f7-f992-4a33-bc3a-136dd2265398\") " pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" Sep 19 06:00:33.581033 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.580884 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x87sr\" (UniqueName: \"kubernetes.io/projected/c90814f7-f992-4a33-bc3a-136dd2265398-kube-api-access-x87sr\") pod \"payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb\" (UID: \"c90814f7-f992-4a33-bc3a-136dd2265398\") " pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" Sep 19 06:00:33.581588 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.581567 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c90814f7-f992-4a33-bc3a-136dd2265398-plugins-config-volume\") pod \"payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb\" (UID: \"c90814f7-f992-4a33-bc3a-136dd2265398\") " pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" Sep 19 06:00:33.588955 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.588928 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x87sr\" (UniqueName: \"kubernetes.io/projected/c90814f7-f992-4a33-bc3a-136dd2265398-kube-api-access-x87sr\") pod \"payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb\" (UID: \"c90814f7-f992-4a33-bc3a-136dd2265398\") " pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" Sep 19 06:00:33.648015 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.647976 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" Sep 19 06:00:33.857388 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.857295 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:33.922181 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:33.922147 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb"] Sep 19 06:00:33.923838 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:00:33.923817 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc90814f7_f992_4a33_bc3a_136dd2265398.slice/crio-253a0601422a06be95fad6284797119c52876282b17cb837a6fefcb4e6bc73de WatchSource:0}: Error finding container 253a0601422a06be95fad6284797119c52876282b17cb837a6fefcb4e6bc73de: Status 404 returned error can't find the container with id 253a0601422a06be95fad6284797119c52876282b17cb837a6fefcb4e6bc73de Sep 19 06:00:34.715124 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:34.715092 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" event={"ID":"c90814f7-f992-4a33-bc3a-136dd2265398","Type":"ContainerStarted","Data":"54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8"} Sep 19 06:00:34.715536 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:34.715141 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" event={"ID":"c90814f7-f992-4a33-bc3a-136dd2265398","Type":"ContainerStarted","Data":"253a0601422a06be95fad6284797119c52876282b17cb837a6fefcb4e6bc73de"} Sep 19 06:00:34.715536 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:34.715189 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" Sep 19 06:00:34.733469 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:34.733410 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" podStartSLOduration=1.7333782009999998 podStartE2EDuration="1.733378201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:00:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:00:34.73260175 +0000 UTC m=+1162.725829277" watchObservedRunningTime="2026-09-19 06:00:34.733378201 +0000 UTC m=+1162.726605732" Sep 19 06:00:45.723886 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:45.723854 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" Sep 19 06:00:48.152248 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:48.152214 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:54.849175 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:54.849133 2584 generic.go:356] "Generic (PLEG): container finished" podID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerID="3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb" exitCode=0 Sep 19 06:00:54.849664 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:54.849206 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" event={"ID":"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3","Type":"ContainerDied","Data":"3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb"} Sep 19 06:00:54.849716 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:54.849704 2584 scope.go:117] "RemoveContainer" containerID="3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb" Sep 19 06:00:54.850972 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:54.850944 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:00:54.851037 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:54.851017 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:00:54.851077 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:54.851050 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:00:55.138763 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:55.138701 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:55.382110 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:55.382077 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:55.858437 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:55.858387 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" event={"ID":"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3","Type":"ContainerStarted","Data":"b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c"} Sep 19 06:00:56.168379 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:56.168295 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:56.513327 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:56.513216 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:56.513536 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:56.513477 2584 patch_prober.go:28] interesting pod/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Sep 19 06:00:56.513608 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:56.513527 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Sep 19 06:00:57.001140 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:57.001107 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:57.489374 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:57.489325 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:57.511677 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:57.511640 2584 patch_prober.go:28] interesting pod/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Sep 19 06:00:57.511871 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:57.511711 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Sep 19 06:00:57.589734 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:57.589702 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:57.662696 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:57.662655 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:00:57.784610 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:57.784489 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:57.934358 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:57.934322 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:58.130205 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.130160 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm"] Sep 19 06:00:58.134670 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.134641 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm" Sep 19 06:00:58.137262 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.137238 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:00:58.138806 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.138780 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/370a61f0-688a-40b9-8ebc-00b39e1060ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm\" (UID: \"370a61f0-688a-40b9-8ebc-00b39e1060ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm" Sep 19 06:00:58.144071 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.144045 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm"] Sep 19 06:00:58.240240 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.240202 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/370a61f0-688a-40b9-8ebc-00b39e1060ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm\" (UID: \"370a61f0-688a-40b9-8ebc-00b39e1060ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm" Sep 19 06:00:58.240876 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.240856 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/370a61f0-688a-40b9-8ebc-00b39e1060ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm\" (UID: \"370a61f0-688a-40b9-8ebc-00b39e1060ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm" Sep 19 06:00:58.448376 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.448285 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm" Sep 19 06:00:58.515564 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.513742 2584 patch_prober.go:28] interesting pod/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Sep 19 06:00:58.515564 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.513803 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Sep 19 06:00:58.742848 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.742822 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm"] Sep 19 06:00:58.745210 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:00:58.745180 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod370a61f0_688a_40b9_8ebc_00b39e1060ab.slice/crio-da089d78b45bfe37e23b560ae1b653463596c8aa1c47b8fa483da060fb2a77cc WatchSource:0}: Error finding container da089d78b45bfe37e23b560ae1b653463596c8aa1c47b8fa483da060fb2a77cc: Status 404 returned error can't find the container with id da089d78b45bfe37e23b560ae1b653463596c8aa1c47b8fa483da060fb2a77cc Sep 19 06:00:58.756724 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.756703 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:58.878527 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:58.878492 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm" event={"ID":"370a61f0-688a-40b9-8ebc-00b39e1060ab","Type":"ContainerStarted","Data":"da089d78b45bfe37e23b560ae1b653463596c8aa1c47b8fa483da060fb2a77cc"} Sep 19 06:00:59.324899 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:59.324861 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:00:59.512246 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:59.512147 2584 patch_prober.go:28] interesting pod/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Sep 19 06:00:59.512532 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:59.512501 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Sep 19 06:00:59.885719 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:59.885685 2584 generic.go:356] "Generic (PLEG): container finished" podID="370a61f0-688a-40b9-8ebc-00b39e1060ab" containerID="76506b862742f0af142b6d41db4a09dd6c69749ed1239e3c7c95bf229a8026c5" exitCode=0 Sep 19 06:00:59.885896 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:00:59.885770 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm" event={"ID":"370a61f0-688a-40b9-8ebc-00b39e1060ab","Type":"ContainerDied","Data":"76506b862742f0af142b6d41db4a09dd6c69749ed1239e3c7c95bf229a8026c5"} Sep 19 06:01:00.142981 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:00.142893 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:00.512750 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:00.512195 2584 patch_prober.go:28] interesting pod/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Sep 19 06:01:00.512750 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:00.512263 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Sep 19 06:01:01.023049 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.023004 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:01.069075 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.069050 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm" Sep 19 06:01:01.171554 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.171457 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/370a61f0-688a-40b9-8ebc-00b39e1060ab-maas-api-service-ca\") pod \"370a61f0-688a-40b9-8ebc-00b39e1060ab\" (UID: \"370a61f0-688a-40b9-8ebc-00b39e1060ab\") " Sep 19 06:01:01.171820 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.171797 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/370a61f0-688a-40b9-8ebc-00b39e1060ab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "370a61f0-688a-40b9-8ebc-00b39e1060ab" (UID: "370a61f0-688a-40b9-8ebc-00b39e1060ab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:01:01.280819 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.280778 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/370a61f0-688a-40b9-8ebc-00b39e1060ab-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:01:01.526880 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.526783 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:01:01.528835 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.528808 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:01:01.563078 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.563047 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:01.658157 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.658130 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:01.899305 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.899275 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm" Sep 19 06:01:01.899500 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.899314 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm" event={"ID":"370a61f0-688a-40b9-8ebc-00b39e1060ab","Type":"ContainerDied","Data":"da089d78b45bfe37e23b560ae1b653463596c8aa1c47b8fa483da060fb2a77cc"} Sep 19 06:01:01.899500 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.899371 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da089d78b45bfe37e23b560ae1b653463596c8aa1c47b8fa483da060fb2a77cc" Sep 19 06:01:01.904436 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:01.904392 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:02.603784 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:02.603752 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:03.410203 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:03.410165 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:04.091232 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:04.091199 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:08.289838 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:08.289807 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:08.907451 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:08.907392 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:08.908750 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:08.908727 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs"] Sep 19 06:01:08.911025 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:08.911004 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c85316-29829960-frsgs"] Sep 19 06:01:08.935120 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:08.935088 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck"] Sep 19 06:01:08.935346 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:08.935322 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" podUID="2623fdd9-d3ce-4cde-9983-2434e6b2e240" containerName="payload-processing" containerID="cri-o://a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a" gracePeriod=30 Sep 19 06:01:09.110143 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.110081 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm"] Sep 19 06:01:09.114349 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.114319 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c85316-9d8cm"] Sep 19 06:01:09.188207 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.188183 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" Sep 19 06:01:09.248335 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.248304 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:09.272567 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.272524 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2623fdd9-d3ce-4cde-9983-2434e6b2e240-plugins-config-volume\") pod \"2623fdd9-d3ce-4cde-9983-2434e6b2e240\" (UID: \"2623fdd9-d3ce-4cde-9983-2434e6b2e240\") " Sep 19 06:01:09.272826 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.272583 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2p6h\" (UniqueName: \"kubernetes.io/projected/2623fdd9-d3ce-4cde-9983-2434e6b2e240-kube-api-access-h2p6h\") pod \"2623fdd9-d3ce-4cde-9983-2434e6b2e240\" (UID: \"2623fdd9-d3ce-4cde-9983-2434e6b2e240\") " Sep 19 06:01:09.273247 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.273217 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2623fdd9-d3ce-4cde-9983-2434e6b2e240-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2623fdd9-d3ce-4cde-9983-2434e6b2e240" (UID: "2623fdd9-d3ce-4cde-9983-2434e6b2e240"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:01:09.275263 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.275229 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2623fdd9-d3ce-4cde-9983-2434e6b2e240-kube-api-access-h2p6h" (OuterVolumeSpecName: "kube-api-access-h2p6h") pod "2623fdd9-d3ce-4cde-9983-2434e6b2e240" (UID: "2623fdd9-d3ce-4cde-9983-2434e6b2e240"). InnerVolumeSpecName "kube-api-access-h2p6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:01:09.310379 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.310327 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:09.373879 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.373839 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2623fdd9-d3ce-4cde-9983-2434e6b2e240-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:01:09.373879 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.373871 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2p6h\" (UniqueName: \"kubernetes.io/projected/2623fdd9-d3ce-4cde-9983-2434e6b2e240-kube-api-access-h2p6h\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:01:09.652857 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.652824 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:09.756701 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.756672 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:09.958953 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.958859 2584 generic.go:356] "Generic (PLEG): container finished" podID="2623fdd9-d3ce-4cde-9983-2434e6b2e240" containerID="a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a" exitCode=0 Sep 19 06:01:09.958953 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.958903 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" Sep 19 06:01:09.959172 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.958949 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" event={"ID":"2623fdd9-d3ce-4cde-9983-2434e6b2e240","Type":"ContainerDied","Data":"a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a"} Sep 19 06:01:09.959172 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.959000 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck" event={"ID":"2623fdd9-d3ce-4cde-9983-2434e6b2e240","Type":"ContainerDied","Data":"5a928e97bc6351662f34ceb6946bb6d33e5d652af8faf8399d8d303ede8c28f6"} Sep 19 06:01:09.959172 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.959022 2584 scope.go:117] "RemoveContainer" containerID="a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a" Sep 19 06:01:09.970307 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.970287 2584 scope.go:117] "RemoveContainer" containerID="a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a" Sep 19 06:01:09.970666 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:01:09.970644 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a\": container with ID starting with a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a not found: ID does not exist" containerID="a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a" Sep 19 06:01:09.970751 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.970679 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a"} err="failed to get container status \"a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a\": rpc error: code = NotFound desc = could not find container \"a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a\": container with ID starting with a8c11428c388daf3ebbfa2709855427e8164f20f6f114bd97db7d1c888c66a4a not found: ID does not exist" Sep 19 06:01:09.984549 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.984517 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck"] Sep 19 06:01:09.988275 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:09.988255 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c85316-7c94d857f9-kkmck"] Sep 19 06:01:10.268008 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:10.267925 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0985253b-4c47-4993-ba51-583328402272" path="/var/lib/kubelet/pods/0985253b-4c47-4993-ba51-583328402272/volumes" Sep 19 06:01:10.268384 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:10.268367 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2623fdd9-d3ce-4cde-9983-2434e6b2e240" path="/var/lib/kubelet/pods/2623fdd9-d3ce-4cde-9983-2434e6b2e240/volumes" Sep 19 06:01:10.268710 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:10.268696 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="370a61f0-688a-40b9-8ebc-00b39e1060ab" path="/var/lib/kubelet/pods/370a61f0-688a-40b9-8ebc-00b39e1060ab/volumes" Sep 19 06:01:10.293524 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:10.293492 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:11.229283 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:11.229241 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:12.123260 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:12.123222 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:12.851006 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:12.850958 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:13.583736 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:13.583683 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:14.176658 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:14.176624 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:14.297394 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:14.297367 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 06:01:14.298828 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:14.298807 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 06:01:14.371984 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:14.371957 2584 scope.go:117] "RemoveContainer" containerID="4a4661cb96f26d1506891e398a8561eed9355b2cb2e3440fded0c6148d03dc34" Sep 19 06:01:15.525520 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:15.525490 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:15.600209 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:15.600178 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:15.808894 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:15.808805 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:16.417776 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:16.417740 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:16.739335 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:16.739248 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:16.934106 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:16.934074 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:19.677628 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:19.677591 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:20.379002 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:20.378966 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:20.494628 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:20.494562 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:21.470239 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:21.470175 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:22.202635 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:22.202603 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:23.223191 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:23.223159 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:23.783920 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:23.783876 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:24.945740 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:24.945704 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:25.059903 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.059876 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:25.161953 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.161921 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:25.214281 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.214203 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt"] Sep 19 06:01:25.214764 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.214748 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="370a61f0-688a-40b9-8ebc-00b39e1060ab" containerName="revoke-keys" Sep 19 06:01:25.214764 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.214764 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="370a61f0-688a-40b9-8ebc-00b39e1060ab" containerName="revoke-keys" Sep 19 06:01:25.214881 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.214777 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2623fdd9-d3ce-4cde-9983-2434e6b2e240" containerName="payload-processing" Sep 19 06:01:25.214881 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.214782 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2623fdd9-d3ce-4cde-9983-2434e6b2e240" containerName="payload-processing" Sep 19 06:01:25.214949 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.214883 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2623fdd9-d3ce-4cde-9983-2434e6b2e240" containerName="payload-processing" Sep 19 06:01:25.214949 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.214895 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="370a61f0-688a-40b9-8ebc-00b39e1060ab" containerName="revoke-keys" Sep 19 06:01:25.219391 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.219372 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" Sep 19 06:01:25.222920 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.222892 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-888f75\"" Sep 19 06:01:25.223049 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.222997 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-888f75-dockercfg-h6j8r\"" Sep 19 06:01:25.232333 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.232308 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt"] Sep 19 06:01:25.339625 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.339591 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75769fa3-8ead-4660-b9ee-3e5d68176c7e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt\" (UID: \"75769fa3-8ead-4660-b9ee-3e5d68176c7e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" Sep 19 06:01:25.339821 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.339668 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prtdc\" (UniqueName: \"kubernetes.io/projected/75769fa3-8ead-4660-b9ee-3e5d68176c7e-kube-api-access-prtdc\") pod \"payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt\" (UID: \"75769fa3-8ead-4660-b9ee-3e5d68176c7e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" Sep 19 06:01:25.440636 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.440596 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75769fa3-8ead-4660-b9ee-3e5d68176c7e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt\" (UID: \"75769fa3-8ead-4660-b9ee-3e5d68176c7e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" Sep 19 06:01:25.440826 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.440725 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prtdc\" (UniqueName: \"kubernetes.io/projected/75769fa3-8ead-4660-b9ee-3e5d68176c7e-kube-api-access-prtdc\") pod \"payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt\" (UID: \"75769fa3-8ead-4660-b9ee-3e5d68176c7e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" Sep 19 06:01:25.441444 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.441388 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75769fa3-8ead-4660-b9ee-3e5d68176c7e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt\" (UID: \"75769fa3-8ead-4660-b9ee-3e5d68176c7e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" Sep 19 06:01:25.449168 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.449147 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prtdc\" (UniqueName: \"kubernetes.io/projected/75769fa3-8ead-4660-b9ee-3e5d68176c7e-kube-api-access-prtdc\") pod \"payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt\" (UID: \"75769fa3-8ead-4660-b9ee-3e5d68176c7e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" Sep 19 06:01:25.530249 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.530152 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" Sep 19 06:01:25.807244 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:25.807216 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt"] Sep 19 06:01:25.808863 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:01:25.808834 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75769fa3_8ead_4660_b9ee_3e5d68176c7e.slice/crio-c9714725b202164b09e8f6c3660798446f5fcaf2cc3756a1edec526239d6f488 WatchSource:0}: Error finding container c9714725b202164b09e8f6c3660798446f5fcaf2cc3756a1edec526239d6f488: Status 404 returned error can't find the container with id c9714725b202164b09e8f6c3660798446f5fcaf2cc3756a1edec526239d6f488 Sep 19 06:01:26.069219 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:26.069125 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" event={"ID":"75769fa3-8ead-4660-b9ee-3e5d68176c7e","Type":"ContainerStarted","Data":"edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4"} Sep 19 06:01:26.069644 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:26.069332 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" Sep 19 06:01:26.069644 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:26.069381 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" event={"ID":"75769fa3-8ead-4660-b9ee-3e5d68176c7e","Type":"ContainerStarted","Data":"c9714725b202164b09e8f6c3660798446f5fcaf2cc3756a1edec526239d6f488"} Sep 19 06:01:26.088442 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:26.088361 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" podStartSLOduration=1.088340592 podStartE2EDuration="1.088340592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:01:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:01:26.087010635 +0000 UTC m=+1214.080238165" watchObservedRunningTime="2026-09-19 06:01:26.088340592 +0000 UTC m=+1214.081568124" Sep 19 06:01:28.483803 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:28.483768 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:31.598843 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:31.598810 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:31.682418 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:31.682363 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9"] Sep 19 06:01:31.686649 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:31.686626 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9" Sep 19 06:01:31.690040 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:31.690019 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:01:31.697041 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:31.697015 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9"] Sep 19 06:01:31.809468 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:31.809421 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e0337ec-a584-42ed-8e10-d9283bd69286-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9\" (UID: \"8e0337ec-a584-42ed-8e10-d9283bd69286\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9" Sep 19 06:01:31.910257 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:31.910155 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e0337ec-a584-42ed-8e10-d9283bd69286-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9\" (UID: \"8e0337ec-a584-42ed-8e10-d9283bd69286\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9" Sep 19 06:01:31.910872 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:31.910849 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e0337ec-a584-42ed-8e10-d9283bd69286-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9\" (UID: \"8e0337ec-a584-42ed-8e10-d9283bd69286\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9" Sep 19 06:01:31.999265 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:31.999232 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9" Sep 19 06:01:32.000707 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:32.000681 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:32.276230 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:32.276204 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9"] Sep 19 06:01:32.278351 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:01:32.278324 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e0337ec_a584_42ed_8e10_d9283bd69286.slice/crio-4f9a95bc7108cd882ffb67fe1a10b3c713f74209389dbbccfee9c88186773d91 WatchSource:0}: Error finding container 4f9a95bc7108cd882ffb67fe1a10b3c713f74209389dbbccfee9c88186773d91: Status 404 returned error can't find the container with id 4f9a95bc7108cd882ffb67fe1a10b3c713f74209389dbbccfee9c88186773d91 Sep 19 06:01:32.868090 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:32.867998 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:33.117980 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:33.117949 2584 generic.go:356] "Generic (PLEG): container finished" podID="8e0337ec-a584-42ed-8e10-d9283bd69286" containerID="8abbaaae1d6cf72b53b40625f3255fc0a22307e4e6ce01c4fe95e9df406b994b" exitCode=0 Sep 19 06:01:33.118196 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:33.118044 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9" event={"ID":"8e0337ec-a584-42ed-8e10-d9283bd69286","Type":"ContainerDied","Data":"8abbaaae1d6cf72b53b40625f3255fc0a22307e4e6ce01c4fe95e9df406b994b"} Sep 19 06:01:33.118196 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:33.118094 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9" event={"ID":"8e0337ec-a584-42ed-8e10-d9283bd69286","Type":"ContainerStarted","Data":"4f9a95bc7108cd882ffb67fe1a10b3c713f74209389dbbccfee9c88186773d91"} Sep 19 06:01:33.720629 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:33.720596 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:34.282081 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:34.282052 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9" Sep 19 06:01:34.339369 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:34.339328 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e0337ec-a584-42ed-8e10-d9283bd69286-maas-api-service-ca\") pod \"8e0337ec-a584-42ed-8e10-d9283bd69286\" (UID: \"8e0337ec-a584-42ed-8e10-d9283bd69286\") " Sep 19 06:01:34.339840 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:34.339805 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e0337ec-a584-42ed-8e10-d9283bd69286-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8e0337ec-a584-42ed-8e10-d9283bd69286" (UID: "8e0337ec-a584-42ed-8e10-d9283bd69286"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:01:34.339962 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:34.339930 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e0337ec-a584-42ed-8e10-d9283bd69286-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:01:34.585747 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:34.585707 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:35.137692 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:35.137662 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9" Sep 19 06:01:35.137692 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:35.137687 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9" event={"ID":"8e0337ec-a584-42ed-8e10-d9283bd69286","Type":"ContainerDied","Data":"4f9a95bc7108cd882ffb67fe1a10b3c713f74209389dbbccfee9c88186773d91"} Sep 19 06:01:35.137909 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:35.137730 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f9a95bc7108cd882ffb67fe1a10b3c713f74209389dbbccfee9c88186773d91" Sep 19 06:01:35.505094 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:35.504984 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:36.467455 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:36.467425 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:36.669797 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:36.669752 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:37.078007 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:37.077977 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" Sep 19 06:01:37.202692 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:37.202656 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:38.005764 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:38.005731 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:38.907761 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:38.907729 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:39.666228 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:39.666202 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:40.202837 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:40.202802 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:40.526868 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:40.526780 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:40.624946 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:40.624906 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:40.917123 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:40.917095 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:41.287475 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:41.287361 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:41.385740 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:41.385712 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:43.809670 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:43.809637 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:44.609739 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:44.609704 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:44.709337 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:44.709309 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:45.506581 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:45.506547 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:46.312214 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:46.312184 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:47.002663 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:47.002634 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:47.901184 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:47.901151 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:48.643290 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:48.643252 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz"] Sep 19 06:01:48.643788 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:48.643553 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" podUID="ebe38f8a-089b-4545-8004-ee46fa95fd1f" containerName="payload-processing" containerID="cri-o://45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5" gracePeriod=30 Sep 19 06:01:48.648143 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:48.648076 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl"] Sep 19 06:01:48.648469 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:48.648439 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" podUID="ccafd6f4-5e73-428b-9b15-6770ac1c9b78" containerName="payload-pre-processing" containerID="cri-o://ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad" gracePeriod=30 Sep 19 06:01:48.687598 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:48.687559 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:48.786158 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:48.786123 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:48.851010 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:48.850975 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9"] Sep 19 06:01:48.857524 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:48.857489 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-cc0e26-ndrm9"] Sep 19 06:01:48.929531 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:48.929505 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" Sep 19 06:01:48.937632 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:48.937593 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:48.954303 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:48.954277 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" Sep 19 06:01:49.039872 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.039842 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:49.105499 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.105467 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccafd6f4-5e73-428b-9b15-6770ac1c9b78-plugins-config-volume\") pod \"ccafd6f4-5e73-428b-9b15-6770ac1c9b78\" (UID: \"ccafd6f4-5e73-428b-9b15-6770ac1c9b78\") " Sep 19 06:01:49.105673 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.105516 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sngt\" (UniqueName: \"kubernetes.io/projected/ebe38f8a-089b-4545-8004-ee46fa95fd1f-kube-api-access-7sngt\") pod \"ebe38f8a-089b-4545-8004-ee46fa95fd1f\" (UID: \"ebe38f8a-089b-4545-8004-ee46fa95fd1f\") " Sep 19 06:01:49.105673 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.105556 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe38f8a-089b-4545-8004-ee46fa95fd1f-plugins-config-volume\") pod \"ebe38f8a-089b-4545-8004-ee46fa95fd1f\" (UID: \"ebe38f8a-089b-4545-8004-ee46fa95fd1f\") " Sep 19 06:01:49.105748 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.105691 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4jzr\" (UniqueName: \"kubernetes.io/projected/ccafd6f4-5e73-428b-9b15-6770ac1c9b78-kube-api-access-g4jzr\") pod \"ccafd6f4-5e73-428b-9b15-6770ac1c9b78\" (UID: \"ccafd6f4-5e73-428b-9b15-6770ac1c9b78\") " Sep 19 06:01:49.105919 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.105886 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccafd6f4-5e73-428b-9b15-6770ac1c9b78-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ccafd6f4-5e73-428b-9b15-6770ac1c9b78" (UID: "ccafd6f4-5e73-428b-9b15-6770ac1c9b78"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:01:49.106019 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.105956 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebe38f8a-089b-4545-8004-ee46fa95fd1f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ebe38f8a-089b-4545-8004-ee46fa95fd1f" (UID: "ebe38f8a-089b-4545-8004-ee46fa95fd1f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:01:49.106114 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.106098 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccafd6f4-5e73-428b-9b15-6770ac1c9b78-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:01:49.106156 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.106116 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe38f8a-089b-4545-8004-ee46fa95fd1f-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:01:49.107754 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.107722 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccafd6f4-5e73-428b-9b15-6770ac1c9b78-kube-api-access-g4jzr" (OuterVolumeSpecName: "kube-api-access-g4jzr") pod "ccafd6f4-5e73-428b-9b15-6770ac1c9b78" (UID: "ccafd6f4-5e73-428b-9b15-6770ac1c9b78"). InnerVolumeSpecName "kube-api-access-g4jzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:01:49.107754 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.107748 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebe38f8a-089b-4545-8004-ee46fa95fd1f-kube-api-access-7sngt" (OuterVolumeSpecName: "kube-api-access-7sngt") pod "ebe38f8a-089b-4545-8004-ee46fa95fd1f" (UID: "ebe38f8a-089b-4545-8004-ee46fa95fd1f"). InnerVolumeSpecName "kube-api-access-7sngt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:01:49.207801 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.207719 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sngt\" (UniqueName: \"kubernetes.io/projected/ebe38f8a-089b-4545-8004-ee46fa95fd1f-kube-api-access-7sngt\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:01:49.207801 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.207744 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4jzr\" (UniqueName: \"kubernetes.io/projected/ccafd6f4-5e73-428b-9b15-6770ac1c9b78-kube-api-access-g4jzr\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:01:49.220860 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.220819 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:49.232340 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.232312 2584 generic.go:356] "Generic (PLEG): container finished" podID="ccafd6f4-5e73-428b-9b15-6770ac1c9b78" containerID="ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad" exitCode=0 Sep 19 06:01:49.232543 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.232358 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" Sep 19 06:01:49.232543 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.232414 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" event={"ID":"ccafd6f4-5e73-428b-9b15-6770ac1c9b78","Type":"ContainerDied","Data":"ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad"} Sep 19 06:01:49.232543 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.232473 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl" event={"ID":"ccafd6f4-5e73-428b-9b15-6770ac1c9b78","Type":"ContainerDied","Data":"8fb437603f7a6605e42515d069bda4fe3386434a65aef22e8f339bc854aeab2f"} Sep 19 06:01:49.232543 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.232506 2584 scope.go:117] "RemoveContainer" containerID="ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad" Sep 19 06:01:49.233923 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.233899 2584 generic.go:356] "Generic (PLEG): container finished" podID="ebe38f8a-089b-4545-8004-ee46fa95fd1f" containerID="45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5" exitCode=0 Sep 19 06:01:49.234039 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.233934 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" event={"ID":"ebe38f8a-089b-4545-8004-ee46fa95fd1f","Type":"ContainerDied","Data":"45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5"} Sep 19 06:01:49.234039 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.233994 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" event={"ID":"ebe38f8a-089b-4545-8004-ee46fa95fd1f","Type":"ContainerDied","Data":"277d26153d3d561c3701f03713bad760c2a418f8daa2ef9706f01f184d707c82"} Sep 19 06:01:49.234152 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.234068 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz" Sep 19 06:01:49.243384 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.243143 2584 scope.go:117] "RemoveContainer" containerID="ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad" Sep 19 06:01:49.243517 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:01:49.243496 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad\": container with ID starting with ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad not found: ID does not exist" containerID="ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad" Sep 19 06:01:49.243588 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.243526 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad"} err="failed to get container status \"ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad\": rpc error: code = NotFound desc = could not find container \"ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad\": container with ID starting with ee23b21e55ff8a1ed2ca857f62986a2b26198f3ad3b7b08e72da781e53ad0aad not found: ID does not exist" Sep 19 06:01:49.243588 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.243545 2584 scope.go:117] "RemoveContainer" containerID="45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5" Sep 19 06:01:49.252844 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.252827 2584 scope.go:117] "RemoveContainer" containerID="45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5" Sep 19 06:01:49.253116 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:01:49.253098 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5\": container with ID starting with 45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5 not found: ID does not exist" containerID="45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5" Sep 19 06:01:49.253161 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.253122 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5"} err="failed to get container status \"45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5\": rpc error: code = NotFound desc = could not find container \"45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5\": container with ID starting with 45e53680dd7d6e7745f54f6768ce7d6e6a4e9645b814cf7796444a8a93a8dcb5 not found: ID does not exist" Sep 19 06:01:49.260379 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.260317 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl"] Sep 19 06:01:49.263040 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.263013 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-cc0e26-57b598b594-l6jrl"] Sep 19 06:01:49.277871 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.277844 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz"] Sep 19 06:01:49.281966 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.281944 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-cc0e26-66bc56769c-hv9wz"] Sep 19 06:01:49.877145 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:49.877111 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:50.001313 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.001201 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:50.267693 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.267608 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e0337ec-a584-42ed-8e10-d9283bd69286" path="/var/lib/kubelet/pods/8e0337ec-a584-42ed-8e10-d9283bd69286/volumes" Sep 19 06:01:50.268009 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.267995 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccafd6f4-5e73-428b-9b15-6770ac1c9b78" path="/var/lib/kubelet/pods/ccafd6f4-5e73-428b-9b15-6770ac1c9b78/volumes" Sep 19 06:01:50.268320 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.268308 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebe38f8a-089b-4545-8004-ee46fa95fd1f" path="/var/lib/kubelet/pods/ebe38f8a-089b-4545-8004-ee46fa95fd1f/volumes" Sep 19 06:01:50.413901 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.413807 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:50.519060 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.518949 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:50.533388 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.533355 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99"] Sep 19 06:01:50.533879 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.533865 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e0337ec-a584-42ed-8e10-d9283bd69286" containerName="revoke-keys" Sep 19 06:01:50.533924 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.533880 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e0337ec-a584-42ed-8e10-d9283bd69286" containerName="revoke-keys" Sep 19 06:01:50.533924 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.533898 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebe38f8a-089b-4545-8004-ee46fa95fd1f" containerName="payload-processing" Sep 19 06:01:50.533924 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.533904 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebe38f8a-089b-4545-8004-ee46fa95fd1f" containerName="payload-processing" Sep 19 06:01:50.533924 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.533916 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccafd6f4-5e73-428b-9b15-6770ac1c9b78" containerName="payload-pre-processing" Sep 19 06:01:50.533924 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.533922 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccafd6f4-5e73-428b-9b15-6770ac1c9b78" containerName="payload-pre-processing" Sep 19 06:01:50.534085 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.534022 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccafd6f4-5e73-428b-9b15-6770ac1c9b78" containerName="payload-pre-processing" Sep 19 06:01:50.534085 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.534032 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebe38f8a-089b-4545-8004-ee46fa95fd1f" containerName="payload-processing" Sep 19 06:01:50.534085 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.534042 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e0337ec-a584-42ed-8e10-d9283bd69286" containerName="revoke-keys" Sep 19 06:01:50.538443 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.538421 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" Sep 19 06:01:50.538925 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.538905 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd"] Sep 19 06:01:50.540844 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.540820 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-f1aad1\"" Sep 19 06:01:50.540988 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.540969 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-f1aad1-dockercfg-zvg7w\"" Sep 19 06:01:50.542518 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.542497 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" Sep 19 06:01:50.549940 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.549912 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99"] Sep 19 06:01:50.557267 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.557239 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd"] Sep 19 06:01:50.722150 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.722111 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b0de816-2634-49c6-94c8-a2c666568195-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99\" (UID: \"6b0de816-2634-49c6-94c8-a2c666568195\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" Sep 19 06:01:50.722348 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.722162 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz27h\" (UniqueName: \"kubernetes.io/projected/6b0de816-2634-49c6-94c8-a2c666568195-kube-api-access-wz27h\") pod \"payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99\" (UID: \"6b0de816-2634-49c6-94c8-a2c666568195\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" Sep 19 06:01:50.722348 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.722258 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d23ae8c-3653-477e-afa0-08ec44c4c13a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd\" (UID: \"2d23ae8c-3653-477e-afa0-08ec44c4c13a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" Sep 19 06:01:50.722487 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.722369 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz5jm\" (UniqueName: \"kubernetes.io/projected/2d23ae8c-3653-477e-afa0-08ec44c4c13a-kube-api-access-lz5jm\") pod \"payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd\" (UID: \"2d23ae8c-3653-477e-afa0-08ec44c4c13a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" Sep 19 06:01:50.806324 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.806242 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:50.823612 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.823570 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz5jm\" (UniqueName: \"kubernetes.io/projected/2d23ae8c-3653-477e-afa0-08ec44c4c13a-kube-api-access-lz5jm\") pod \"payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd\" (UID: \"2d23ae8c-3653-477e-afa0-08ec44c4c13a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" Sep 19 06:01:50.823810 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.823663 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b0de816-2634-49c6-94c8-a2c666568195-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99\" (UID: \"6b0de816-2634-49c6-94c8-a2c666568195\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" Sep 19 06:01:50.823810 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.823692 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz27h\" (UniqueName: \"kubernetes.io/projected/6b0de816-2634-49c6-94c8-a2c666568195-kube-api-access-wz27h\") pod \"payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99\" (UID: \"6b0de816-2634-49c6-94c8-a2c666568195\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" Sep 19 06:01:50.823810 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.823744 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d23ae8c-3653-477e-afa0-08ec44c4c13a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd\" (UID: \"2d23ae8c-3653-477e-afa0-08ec44c4c13a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" Sep 19 06:01:50.824416 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.824375 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b0de816-2634-49c6-94c8-a2c666568195-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99\" (UID: \"6b0de816-2634-49c6-94c8-a2c666568195\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" Sep 19 06:01:50.824528 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.824458 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d23ae8c-3653-477e-afa0-08ec44c4c13a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd\" (UID: \"2d23ae8c-3653-477e-afa0-08ec44c4c13a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" Sep 19 06:01:50.835152 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.835116 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz5jm\" (UniqueName: \"kubernetes.io/projected/2d23ae8c-3653-477e-afa0-08ec44c4c13a-kube-api-access-lz5jm\") pod \"payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd\" (UID: \"2d23ae8c-3653-477e-afa0-08ec44c4c13a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" Sep 19 06:01:50.835272 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.835183 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz27h\" (UniqueName: \"kubernetes.io/projected/6b0de816-2634-49c6-94c8-a2c666568195-kube-api-access-wz27h\") pod \"payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99\" (UID: \"6b0de816-2634-49c6-94c8-a2c666568195\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" Sep 19 06:01:50.851201 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.851168 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" Sep 19 06:01:50.859610 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:50.859585 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" Sep 19 06:01:51.130079 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:51.130055 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99"] Sep 19 06:01:51.132127 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:01:51.132091 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b0de816_2634_49c6_94c8_a2c666568195.slice/crio-0fb87eee569a8276acc085f04d7589dc0160a708a5475ffd5e273eb139c3ed0e WatchSource:0}: Error finding container 0fb87eee569a8276acc085f04d7589dc0160a708a5475ffd5e273eb139c3ed0e: Status 404 returned error can't find the container with id 0fb87eee569a8276acc085f04d7589dc0160a708a5475ffd5e273eb139c3ed0e Sep 19 06:01:51.148038 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:51.147999 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd"] Sep 19 06:01:51.151017 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:01:51.150992 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d23ae8c_3653_477e_afa0_08ec44c4c13a.slice/crio-07161157b1ef73a91d9a0fe72716059c53a0f7d6d494a79682c0ac28c5133f16 WatchSource:0}: Error finding container 07161157b1ef73a91d9a0fe72716059c53a0f7d6d494a79682c0ac28c5133f16: Status 404 returned error can't find the container with id 07161157b1ef73a91d9a0fe72716059c53a0f7d6d494a79682c0ac28c5133f16 Sep 19 06:01:51.253004 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:51.252969 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" event={"ID":"6b0de816-2634-49c6-94c8-a2c666568195","Type":"ContainerStarted","Data":"3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629"} Sep 19 06:01:51.253202 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:51.253031 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" event={"ID":"6b0de816-2634-49c6-94c8-a2c666568195","Type":"ContainerStarted","Data":"0fb87eee569a8276acc085f04d7589dc0160a708a5475ffd5e273eb139c3ed0e"} Sep 19 06:01:51.253202 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:51.253148 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" Sep 19 06:01:51.254630 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:51.254605 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" event={"ID":"2d23ae8c-3653-477e-afa0-08ec44c4c13a","Type":"ContainerStarted","Data":"a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41"} Sep 19 06:01:51.254750 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:51.254644 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" event={"ID":"2d23ae8c-3653-477e-afa0-08ec44c4c13a","Type":"ContainerStarted","Data":"07161157b1ef73a91d9a0fe72716059c53a0f7d6d494a79682c0ac28c5133f16"} Sep 19 06:01:51.254750 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:51.254679 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" Sep 19 06:01:51.273202 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:51.273137 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" podStartSLOduration=1.2731179 podStartE2EDuration="1.2731179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:01:51.271810394 +0000 UTC m=+1239.265037926" watchObservedRunningTime="2026-09-19 06:01:51.2731179 +0000 UTC m=+1239.266345433" Sep 19 06:01:51.289599 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:51.289546 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" podStartSLOduration=1.2895276980000001 podStartE2EDuration="1.289527698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:01:51.288140049 +0000 UTC m=+1239.281367581" watchObservedRunningTime="2026-09-19 06:01:51.289527698 +0000 UTC m=+1239.282755270" Sep 19 06:01:51.625260 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:51.625222 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:52.392945 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:52.392917 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:52.999631 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:52.999601 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:53.420829 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:53.420791 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:54.192304 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:54.192272 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:54.901257 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:54.901222 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:55.353354 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:55.353316 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:55.452278 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:55.452245 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:55.622158 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:55.622074 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r"] Sep 19 06:01:55.622348 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:55.622319 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" containerID="cri-o://e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082" gracePeriod=30 Sep 19 06:01:55.645380 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:55.645354 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:55.743678 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:55.743645 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:01:56.360869 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:56.360825 2584 patch_prober.go:28] interesting pod/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 19 06:01:56.361368 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:56.360899 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 19 06:01:57.235683 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:01:57.235644 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:00.878016 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.877989 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 06:02:00.929604 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.929531 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d772db8e-bc76-4627-862c-f7588ae39638-istio-podinfo\") pod \"d772db8e-bc76-4627-862c-f7588ae39638\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " Sep 19 06:02:00.929604 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.929578 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-istio-envoy\") pod \"d772db8e-bc76-4627-862c-f7588ae39638\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " Sep 19 06:02:00.929790 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.929625 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-workload-certs\") pod \"d772db8e-bc76-4627-862c-f7588ae39638\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " Sep 19 06:02:00.929790 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.929646 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2w5n4\" (UniqueName: \"kubernetes.io/projected/d772db8e-bc76-4627-862c-f7588ae39638-kube-api-access-2w5n4\") pod \"d772db8e-bc76-4627-862c-f7588ae39638\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " Sep 19 06:02:00.929790 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.929698 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d772db8e-bc76-4627-862c-f7588ae39638-istio-token\") pod \"d772db8e-bc76-4627-862c-f7588ae39638\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " Sep 19 06:02:00.929790 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.929716 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d772db8e-bc76-4627-862c-f7588ae39638-istiod-ca-cert\") pod \"d772db8e-bc76-4627-862c-f7588ae39638\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " Sep 19 06:02:00.929790 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.929762 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-workload-socket\") pod \"d772db8e-bc76-4627-862c-f7588ae39638\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " Sep 19 06:02:00.930067 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.929798 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-credential-socket\") pod \"d772db8e-bc76-4627-862c-f7588ae39638\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " Sep 19 06:02:00.930067 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.929842 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-istio-data\") pod \"d772db8e-bc76-4627-862c-f7588ae39638\" (UID: \"d772db8e-bc76-4627-862c-f7588ae39638\") " Sep 19 06:02:00.930067 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.929951 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d772db8e-bc76-4627-862c-f7588ae39638" (UID: "d772db8e-bc76-4627-862c-f7588ae39638"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:02:00.930237 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.930158 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d772db8e-bc76-4627-862c-f7588ae39638" (UID: "d772db8e-bc76-4627-862c-f7588ae39638"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:02:00.930353 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.930334 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-workload-certs\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:02:00.930542 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.930526 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-workload-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:02:00.930637 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.930339 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d772db8e-bc76-4627-862c-f7588ae39638" (UID: "d772db8e-bc76-4627-862c-f7588ae39638"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:02:00.930728 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.930379 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-istio-data" (OuterVolumeSpecName: "istio-data") pod "d772db8e-bc76-4627-862c-f7588ae39638" (UID: "d772db8e-bc76-4627-862c-f7588ae39638"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:02:00.930836 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.930767 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d772db8e-bc76-4627-862c-f7588ae39638-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d772db8e-bc76-4627-862c-f7588ae39638" (UID: "d772db8e-bc76-4627-862c-f7588ae39638"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:02:00.932067 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.932024 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d772db8e-bc76-4627-862c-f7588ae39638-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d772db8e-bc76-4627-862c-f7588ae39638" (UID: "d772db8e-bc76-4627-862c-f7588ae39638"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 06:02:00.932067 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.932053 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d772db8e-bc76-4627-862c-f7588ae39638-kube-api-access-2w5n4" (OuterVolumeSpecName: "kube-api-access-2w5n4") pod "d772db8e-bc76-4627-862c-f7588ae39638" (UID: "d772db8e-bc76-4627-862c-f7588ae39638"). InnerVolumeSpecName "kube-api-access-2w5n4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:02:00.932203 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.932101 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d772db8e-bc76-4627-862c-f7588ae39638-istio-token" (OuterVolumeSpecName: "istio-token") pod "d772db8e-bc76-4627-862c-f7588ae39638" (UID: "d772db8e-bc76-4627-862c-f7588ae39638"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:02:00.932320 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:00.932301 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d772db8e-bc76-4627-862c-f7588ae39638" (UID: "d772db8e-bc76-4627-862c-f7588ae39638"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:02:01.031839 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.031804 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-credential-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:02:01.031839 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.031834 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-istio-data\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:02:01.031839 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.031842 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d772db8e-bc76-4627-862c-f7588ae39638-istio-podinfo\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:02:01.031839 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.031850 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d772db8e-bc76-4627-862c-f7588ae39638-istio-envoy\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:02:01.032102 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.031858 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w5n4\" (UniqueName: \"kubernetes.io/projected/d772db8e-bc76-4627-862c-f7588ae39638-kube-api-access-2w5n4\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:02:01.032102 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.031867 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d772db8e-bc76-4627-862c-f7588ae39638-istio-token\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:02:01.032102 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.031875 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d772db8e-bc76-4627-862c-f7588ae39638-istiod-ca-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:02:01.317258 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.317172 2584 generic.go:356] "Generic (PLEG): container finished" podID="d772db8e-bc76-4627-862c-f7588ae39638" containerID="e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082" exitCode=0 Sep 19 06:02:01.317258 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.317224 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" Sep 19 06:02:01.317471 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.317262 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" event={"ID":"d772db8e-bc76-4627-862c-f7588ae39638","Type":"ContainerDied","Data":"e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082"} Sep 19 06:02:01.317471 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.317325 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r" event={"ID":"d772db8e-bc76-4627-862c-f7588ae39638","Type":"ContainerDied","Data":"d7285a937ec02121cef885d53c90b92bbebbaefabf40a41ebb6da0555c3b48f5"} Sep 19 06:02:01.317471 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.317354 2584 scope.go:117] "RemoveContainer" containerID="e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082" Sep 19 06:02:01.328613 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.328593 2584 scope.go:117] "RemoveContainer" containerID="60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a" Sep 19 06:02:01.339321 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.339293 2584 scope.go:117] "RemoveContainer" containerID="e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082" Sep 19 06:02:01.339712 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:02:01.339691 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082\": container with ID starting with e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082 not found: ID does not exist" containerID="e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082" Sep 19 06:02:01.339822 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.339724 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082"} err="failed to get container status \"e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082\": rpc error: code = NotFound desc = could not find container \"e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082\": container with ID starting with e7ce6225fac299caaf2dd53ce8b3c6b9f6d3db00ca7568ecf822a25e8f337082 not found: ID does not exist" Sep 19 06:02:01.339822 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.339750 2584 scope.go:117] "RemoveContainer" containerID="60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a" Sep 19 06:02:01.340079 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:02:01.340061 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a\": container with ID starting with 60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a not found: ID does not exist" containerID="60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a" Sep 19 06:02:01.340120 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.340086 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a"} err="failed to get container status \"60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a\": rpc error: code = NotFound desc = could not find container \"60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a\": container with ID starting with 60edceed3b34ae27b8e8b82e737b576b4c919bc141f7e360510b9087a440bf2a not found: ID does not exist" Sep 19 06:02:01.341668 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.341647 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r"] Sep 19 06:02:01.348210 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:01.348187 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-cc0e26-openshift-default-5d7cf6d7c8-l4z2r"] Sep 19 06:02:02.268677 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:02.268640 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d772db8e-bc76-4627-862c-f7588ae39638" path="/var/lib/kubelet/pods/d772db8e-bc76-4627-862c-f7588ae39638/volumes" Sep 19 06:02:02.269051 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:02.268979 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" Sep 19 06:02:02.269051 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:02.269027 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" Sep 19 06:02:07.192623 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:07.192590 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:07.410628 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:07.410591 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:07.872809 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:07.872772 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:14.429367 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:14.429334 2584 scope.go:117] "RemoveContainer" containerID="f82cf4c0e97719114316f22056c6c6f6282d8fa30dd4f0eb2343d837e12e8f7e" Sep 19 06:02:14.438572 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:14.438547 2584 scope.go:117] "RemoveContainer" containerID="bbb472763a13d3fac0acc3bf1b862da0572ab5d465727d5eb914ee99040357e6" Sep 19 06:02:14.447198 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:14.447172 2584 scope.go:117] "RemoveContainer" containerID="bdfb09263db38b08b68c67da69509bb42606cafee6c17d72ffe3a135a7c1a583" Sep 19 06:02:17.576609 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:17.576570 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:18.193331 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:18.193302 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:19.101689 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:19.101657 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:19.686517 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:19.686487 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:19.788639 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:19.788599 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:20.592608 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:20.592574 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:21.397570 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:21.397533 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:21.929462 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:21.929430 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:22.405012 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:22.404978 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:27.198123 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:27.198092 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:39.491156 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:39.491116 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:40.764489 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.764452 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h"] Sep 19 06:02:40.764978 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.764964 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" Sep 19 06:02:40.765028 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.764980 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" Sep 19 06:02:40.765028 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.764994 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" Sep 19 06:02:40.765028 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.764999 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" Sep 19 06:02:40.765125 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.765063 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" Sep 19 06:02:40.768311 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.768295 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h" Sep 19 06:02:40.771635 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.771611 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:02:40.778125 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.778092 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h"] Sep 19 06:02:40.846725 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.846690 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa8c81b-82ea-4bc2-b824-ff16dc858d58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-957d1190-jhz2h\" (UID: \"caa8c81b-82ea-4bc2-b824-ff16dc858d58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h" Sep 19 06:02:40.947811 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.947772 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa8c81b-82ea-4bc2-b824-ff16dc858d58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-957d1190-jhz2h\" (UID: \"caa8c81b-82ea-4bc2-b824-ff16dc858d58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h" Sep 19 06:02:40.948442 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.948393 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa8c81b-82ea-4bc2-b824-ff16dc858d58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-957d1190-jhz2h\" (UID: \"caa8c81b-82ea-4bc2-b824-ff16dc858d58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h" Sep 19 06:02:40.966817 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:40.966785 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:41.081064 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:41.081038 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h" Sep 19 06:02:41.357010 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:41.356981 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h"] Sep 19 06:02:41.357950 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:02:41.357923 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaa8c81b_82ea_4bc2_b824_ff16dc858d58.slice/crio-79cb81273774607dc837409c036d8d94fb75b48fea6bb7074ee285747e0baf71 WatchSource:0}: Error finding container 79cb81273774607dc837409c036d8d94fb75b48fea6bb7074ee285747e0baf71: Status 404 returned error can't find the container with id 79cb81273774607dc837409c036d8d94fb75b48fea6bb7074ee285747e0baf71 Sep 19 06:02:41.570856 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:41.570822 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h" event={"ID":"caa8c81b-82ea-4bc2-b824-ff16dc858d58","Type":"ContainerStarted","Data":"79cb81273774607dc837409c036d8d94fb75b48fea6bb7074ee285747e0baf71"} Sep 19 06:02:41.782821 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:41.782789 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:42.459583 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:42.459555 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:42.578487 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:42.578446 2584 generic.go:356] "Generic (PLEG): container finished" podID="caa8c81b-82ea-4bc2-b824-ff16dc858d58" containerID="0f33b817e9ec2245b3bcd32e6178788f3718d40af9af7310fdc1aff3d22aab57" exitCode=0 Sep 19 06:02:42.578662 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:42.578492 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h" event={"ID":"caa8c81b-82ea-4bc2-b824-ff16dc858d58","Type":"ContainerDied","Data":"0f33b817e9ec2245b3bcd32e6178788f3718d40af9af7310fdc1aff3d22aab57"} Sep 19 06:02:42.590863 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:42.590834 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:43.276770 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:43.276734 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:43.740243 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:43.740211 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h" Sep 19 06:02:43.880452 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:43.880334 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa8c81b-82ea-4bc2-b824-ff16dc858d58-maas-api-service-ca\") pod \"caa8c81b-82ea-4bc2-b824-ff16dc858d58\" (UID: \"caa8c81b-82ea-4bc2-b824-ff16dc858d58\") " Sep 19 06:02:43.880747 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:43.880723 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caa8c81b-82ea-4bc2-b824-ff16dc858d58-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "caa8c81b-82ea-4bc2-b824-ff16dc858d58" (UID: "caa8c81b-82ea-4bc2-b824-ff16dc858d58"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:02:43.982189 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:43.982155 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa8c81b-82ea-4bc2-b824-ff16dc858d58-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:02:43.999579 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:43.999543 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:44.600340 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:44.600300 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h" event={"ID":"caa8c81b-82ea-4bc2-b824-ff16dc858d58","Type":"ContainerDied","Data":"79cb81273774607dc837409c036d8d94fb75b48fea6bb7074ee285747e0baf71"} Sep 19 06:02:44.600340 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:44.600331 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h" Sep 19 06:02:44.600340 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:44.600349 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79cb81273774607dc837409c036d8d94fb75b48fea6bb7074ee285747e0baf71" Sep 19 06:02:44.686812 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:44.686778 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:45.317231 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:45.317201 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:45.981932 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:45.981897 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:56.259678 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.259647 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:56.290124 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.290088 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb"] Sep 19 06:02:56.290386 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.290357 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" podUID="c90814f7-f992-4a33-bc3a-136dd2265398" containerName="payload-processing" containerID="cri-o://54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8" gracePeriod=30 Sep 19 06:02:56.366436 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.366389 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:56.445978 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.445940 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h"] Sep 19 06:02:56.447269 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.447173 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:56.448809 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.448784 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-957d1190-jhz2h"] Sep 19 06:02:56.502669 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.502634 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:56.558505 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.558477 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" Sep 19 06:02:56.674786 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.674755 2584 generic.go:356] "Generic (PLEG): container finished" podID="c90814f7-f992-4a33-bc3a-136dd2265398" containerID="54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8" exitCode=0 Sep 19 06:02:56.674978 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.674809 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" Sep 19 06:02:56.674978 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.674827 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" event={"ID":"c90814f7-f992-4a33-bc3a-136dd2265398","Type":"ContainerDied","Data":"54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8"} Sep 19 06:02:56.674978 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.674882 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb" event={"ID":"c90814f7-f992-4a33-bc3a-136dd2265398","Type":"ContainerDied","Data":"253a0601422a06be95fad6284797119c52876282b17cb837a6fefcb4e6bc73de"} Sep 19 06:02:56.674978 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.674909 2584 scope.go:117] "RemoveContainer" containerID="54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8" Sep 19 06:02:56.684444 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.684387 2584 scope.go:117] "RemoveContainer" containerID="54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8" Sep 19 06:02:56.684702 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:02:56.684682 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8\": container with ID starting with 54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8 not found: ID does not exist" containerID="54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8" Sep 19 06:02:56.684754 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.684714 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8"} err="failed to get container status \"54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8\": rpc error: code = NotFound desc = could not find container \"54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8\": container with ID starting with 54a56aec4fbe812d958ea9e802998d3ace49c907373c753e271c095cc413fbd8 not found: ID does not exist" Sep 19 06:02:56.714233 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.714202 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c90814f7-f992-4a33-bc3a-136dd2265398-plugins-config-volume\") pod \"c90814f7-f992-4a33-bc3a-136dd2265398\" (UID: \"c90814f7-f992-4a33-bc3a-136dd2265398\") " Sep 19 06:02:56.714233 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.714236 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x87sr\" (UniqueName: \"kubernetes.io/projected/c90814f7-f992-4a33-bc3a-136dd2265398-kube-api-access-x87sr\") pod \"c90814f7-f992-4a33-bc3a-136dd2265398\" (UID: \"c90814f7-f992-4a33-bc3a-136dd2265398\") " Sep 19 06:02:56.714655 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.714632 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c90814f7-f992-4a33-bc3a-136dd2265398-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c90814f7-f992-4a33-bc3a-136dd2265398" (UID: "c90814f7-f992-4a33-bc3a-136dd2265398"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:02:56.716391 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.716366 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c90814f7-f992-4a33-bc3a-136dd2265398-kube-api-access-x87sr" (OuterVolumeSpecName: "kube-api-access-x87sr") pod "c90814f7-f992-4a33-bc3a-136dd2265398" (UID: "c90814f7-f992-4a33-bc3a-136dd2265398"). InnerVolumeSpecName "kube-api-access-x87sr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:02:56.815425 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.815323 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c90814f7-f992-4a33-bc3a-136dd2265398-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:02:56.815425 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.815355 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x87sr\" (UniqueName: \"kubernetes.io/projected/c90814f7-f992-4a33-bc3a-136dd2265398-kube-api-access-x87sr\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:02:56.885343 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.885308 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:56.997819 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:56.997790 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb"] Sep 19 06:02:57.001453 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:57.001427 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-957d1190-6bc8475fdf-b24wb"] Sep 19 06:02:57.542725 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:57.542694 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:58.268098 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:58.268063 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c90814f7-f992-4a33-bc3a-136dd2265398" path="/var/lib/kubelet/pods/c90814f7-f992-4a33-bc3a-136dd2265398/volumes" Sep 19 06:02:58.268445 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:58.268431 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caa8c81b-82ea-4bc2-b824-ff16dc858d58" path="/var/lib/kubelet/pods/caa8c81b-82ea-4bc2-b824-ff16dc858d58/volumes" Sep 19 06:02:58.274355 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:58.274322 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:58.918062 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:58.918025 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:59.538373 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:59.538339 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:02:59.703443 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:02:59.703414 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:03:00.287648 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:03:00.287617 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:03:04.723433 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:03:04.723385 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:03:24.272423 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:03:24.272374 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:03:41.825057 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:03:41.825020 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:03:41.972511 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:03:41.972476 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:03:47.390946 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:03:47.390904 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:03:51.275703 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:03:51.275614 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:03:58.983921 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:03:58.983889 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:10.125424 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:10.125371 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:10.210628 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:10.210597 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:10.369533 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:10.369497 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq"] Sep 19 06:04:10.369791 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:10.369765 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerName="istio-proxy" containerID="cri-o://b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c" gracePeriod=30 Sep 19 06:04:10.383056 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:10.382986 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:11.697330 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:11.697285 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:11.795069 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:11.795033 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:11.895253 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:11.895218 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:12.495332 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:12.495295 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:12.594849 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:12.594821 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:12.694800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:12.694762 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:13.999693 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:13.999660 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:14.506527 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:14.506491 2584 scope.go:117] "RemoveContainer" containerID="6860303196e04421b03c28ca4b5b9adc0e1ca228c9c3644c72954c9da5d47630" Sep 19 06:04:15.628819 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.628792 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:04:15.726704 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.726610 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-podinfo\") pod \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " Sep 19 06:04:15.726704 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.726679 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-envoy\") pod \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " Sep 19 06:04:15.726966 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.726716 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-workload-certs\") pod \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " Sep 19 06:04:15.726966 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.726765 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-credential-socket\") pod \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " Sep 19 06:04:15.726966 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.726792 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-data\") pod \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " Sep 19 06:04:15.726966 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.726827 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-token\") pod \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " Sep 19 06:04:15.726966 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.726886 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-workload-socket\") pod \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " Sep 19 06:04:15.726966 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.726918 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2dr2\" (UniqueName: \"kubernetes.io/projected/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-kube-api-access-v2dr2\") pod \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " Sep 19 06:04:15.726966 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.726956 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istiod-ca-cert\") pod \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\" (UID: \"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3\") " Sep 19 06:04:15.727381 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.726980 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" (UID: "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:04:15.727381 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.727119 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-data" (OuterVolumeSpecName: "istio-data") pod "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" (UID: "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:04:15.727381 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.727235 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" (UID: "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:04:15.727381 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.727248 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" (UID: "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:04:15.727639 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.727467 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-workload-certs\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:04:15.727639 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.727495 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-credential-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:04:15.727639 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.727510 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-data\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:04:15.727639 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.727522 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-workload-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:04:15.727639 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.727575 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" (UID: "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:04:15.729112 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.729087 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" (UID: "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:04:15.729112 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.729094 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" (UID: "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 06:04:15.729272 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.729143 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-kube-api-access-v2dr2" (OuterVolumeSpecName: "kube-api-access-v2dr2") pod "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" (UID: "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3"). InnerVolumeSpecName "kube-api-access-v2dr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:04:15.729272 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.729163 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-token" (OuterVolumeSpecName: "istio-token") pod "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" (UID: "21dcd431-5ee8-4a82-bf15-71e9fd9b79a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:04:15.828859 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.828825 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-envoy\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:04:15.828859 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.828857 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-token\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:04:15.829073 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.828869 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2dr2\" (UniqueName: \"kubernetes.io/projected/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-kube-api-access-v2dr2\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:04:15.829073 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.828883 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istiod-ca-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:04:15.829073 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:15.828894 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3-istio-podinfo\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:04:16.160103 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.160063 2584 generic.go:356] "Generic (PLEG): container finished" podID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerID="b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c" exitCode=0 Sep 19 06:04:16.160282 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.160117 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" Sep 19 06:04:16.160282 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.160119 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" event={"ID":"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3","Type":"ContainerDied","Data":"b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c"} Sep 19 06:04:16.160282 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.160197 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq" event={"ID":"21dcd431-5ee8-4a82-bf15-71e9fd9b79a3","Type":"ContainerDied","Data":"54d0ff9279c90c56cebb6b82869fe4957c191b85fa914974ba89e583d5235a96"} Sep 19 06:04:16.160282 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.160222 2584 scope.go:117] "RemoveContainer" containerID="b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c" Sep 19 06:04:16.170688 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.170668 2584 scope.go:117] "RemoveContainer" containerID="3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb" Sep 19 06:04:16.179458 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.179440 2584 scope.go:117] "RemoveContainer" containerID="b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c" Sep 19 06:04:16.179734 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:04:16.179717 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c\": container with ID starting with b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c not found: ID does not exist" containerID="b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c" Sep 19 06:04:16.179808 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.179742 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c"} err="failed to get container status \"b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c\": rpc error: code = NotFound desc = could not find container \"b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c\": container with ID starting with b003beed2ed325e6048901b30227e26440959c2758ee87b5dac9062e7d26d72c not found: ID does not exist" Sep 19 06:04:16.179808 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.179759 2584 scope.go:117] "RemoveContainer" containerID="3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb" Sep 19 06:04:16.179996 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:04:16.179976 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb\": container with ID starting with 3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb not found: ID does not exist" containerID="3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb" Sep 19 06:04:16.180038 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.180003 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb"} err="failed to get container status \"3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb\": rpc error: code = NotFound desc = could not find container \"3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb\": container with ID starting with 3c5812be723a897e56525a37116f6de942f6ab9c71ac0fdfabf360e1592034bb not found: ID does not exist" Sep 19 06:04:16.194548 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.194521 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq"] Sep 19 06:04:16.199078 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.199056 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-957d1190-openshift-default-f6c658df6-qpvlq"] Sep 19 06:04:16.268390 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.268358 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" path="/var/lib/kubelet/pods/21dcd431-5ee8-4a82-bf15-71e9fd9b79a3/volumes" Sep 19 06:04:16.767849 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:16.767815 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:18.876391 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:18.876337 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:18.977294 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:18.977259 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:19.669395 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:19.669361 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:19.769721 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:19.769694 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:20.348824 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:20.348786 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:26.053565 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:26.053533 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:26.172933 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:26.172877 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:26.273509 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:26.273467 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:35.549148 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:35.549116 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:36.303156 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:36.303117 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:36.769725 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:36.769685 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:37.002030 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:37.002001 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:52.486810 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:52.486775 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:04:57.999838 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:04:57.999756 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:03.578158 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:03.578127 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:03.677806 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:03.677774 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:03.941131 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:03.941082 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:04.217292 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.217210 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx"] Sep 19 06:05:04.217955 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.217938 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerName="istio-proxy" Sep 19 06:05:04.218011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.217957 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerName="istio-proxy" Sep 19 06:05:04.218011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.217968 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caa8c81b-82ea-4bc2-b824-ff16dc858d58" containerName="revoke-keys" Sep 19 06:05:04.218011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.217973 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="caa8c81b-82ea-4bc2-b824-ff16dc858d58" containerName="revoke-keys" Sep 19 06:05:04.218011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.217987 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c90814f7-f992-4a33-bc3a-136dd2265398" containerName="payload-processing" Sep 19 06:05:04.218011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.217994 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c90814f7-f992-4a33-bc3a-136dd2265398" containerName="payload-processing" Sep 19 06:05:04.218011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.218000 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerName="istio-proxy" Sep 19 06:05:04.218011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.218007 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerName="istio-proxy" Sep 19 06:05:04.218244 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.218100 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerName="istio-proxy" Sep 19 06:05:04.218244 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.218107 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="caa8c81b-82ea-4bc2-b824-ff16dc858d58" containerName="revoke-keys" Sep 19 06:05:04.218244 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.218116 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d772db8e-bc76-4627-862c-f7588ae39638" containerName="istio-proxy" Sep 19 06:05:04.218244 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.218123 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="21dcd431-5ee8-4a82-bf15-71e9fd9b79a3" containerName="istio-proxy" Sep 19 06:05:04.218244 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.218134 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c90814f7-f992-4a33-bc3a-136dd2265398" containerName="payload-processing" Sep 19 06:05:04.220239 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.220221 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.222611 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.222591 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-311e2d-openshift-default-dockercfg-gnn2d\"" Sep 19 06:05:04.231247 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.231215 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx"] Sep 19 06:05:04.244344 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.244314 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:04.322713 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.322671 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-podinfo\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.322713 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.322718 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-credential-socket\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.322943 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.322778 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-envoy\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.322943 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.322825 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-workload-socket\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.322943 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.322859 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-workload-certs\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.322943 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.322926 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istiod-ca-cert\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.323073 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.322944 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmfph\" (UniqueName: \"kubernetes.io/projected/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-kube-api-access-qmfph\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.323073 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.322971 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-data\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.323073 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.322993 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-token\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.401882 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.401847 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:04.424100 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424061 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-data\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.424100 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424101 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-token\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.424386 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424141 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-podinfo\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.424386 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424165 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-credential-socket\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.424386 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424193 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-envoy\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.424386 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424226 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-workload-socket\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.424660 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424586 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-data\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.424660 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424618 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-credential-socket\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.424775 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424697 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-workload-socket\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.424775 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424697 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-workload-certs\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.424775 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424764 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istiod-ca-cert\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.424911 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424788 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmfph\" (UniqueName: \"kubernetes.io/projected/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-kube-api-access-qmfph\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.424911 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.424799 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-workload-certs\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.425376 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.425358 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istiod-ca-cert\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.426591 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.426569 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-envoy\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.426726 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.426709 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-podinfo\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.431988 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.431964 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-token\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.432091 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.432026 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmfph\" (UniqueName: \"kubernetes.io/projected/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-kube-api-access-qmfph\") pod \"e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.532935 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.532846 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:04.808802 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.808770 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx"] Sep 19 06:05:04.809887 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:05:04.809858 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc2b311_d4a2_4cd8_97f6_b6f8fc147514.slice/crio-9279ff376c7a9be44f4eb52474beffd1e4615bc1766fcc90371b30cbcad633c3 WatchSource:0}: Error finding container 9279ff376c7a9be44f4eb52474beffd1e4615bc1766fcc90371b30cbcad633c3: Status 404 returned error can't find the container with id 9279ff376c7a9be44f4eb52474beffd1e4615bc1766fcc90371b30cbcad633c3 Sep 19 06:05:04.812271 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.812225 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:05:04.812346 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.812322 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:05:04.812385 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:04.812366 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:05:05.452149 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:05.452115 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" event={"ID":"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514","Type":"ContainerStarted","Data":"60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a"} Sep 19 06:05:05.452343 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:05.452172 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" event={"ID":"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514","Type":"ContainerStarted","Data":"9279ff376c7a9be44f4eb52474beffd1e4615bc1766fcc90371b30cbcad633c3"} Sep 19 06:05:05.472258 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:05.472205 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" podStartSLOduration=1.4721832080000001 podStartE2EDuration="1.472183208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:05:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:05:05.470257463 +0000 UTC m=+1433.463484994" watchObservedRunningTime="2026-09-19 06:05:05.472183208 +0000 UTC m=+1433.465410799" Sep 19 06:05:05.533269 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:05.533232 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:05.539322 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:05.539293 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:06.458960 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:06.458932 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:06.459996 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:06.459971 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:06.803035 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:06.802945 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:07.515907 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:07.515876 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:07.612832 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:07.612803 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:08.508830 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:08.508797 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:09.239101 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:09.239046 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:10.085895 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:10.085838 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:10.635683 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:10.635646 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:11.319315 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.319279 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:11.378048 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.377995 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:11.480238 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.480204 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:11.769837 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.769805 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:11.864613 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.864584 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:11.869224 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.869187 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw"] Sep 19 06:05:11.873193 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.873175 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" Sep 19 06:05:11.875750 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.875729 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-311e2d-dockercfg-htnh5\"" Sep 19 06:05:11.875870 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.875764 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-311e2d\"" Sep 19 06:05:11.884563 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.884529 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw"] Sep 19 06:05:11.892259 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.892230 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn"] Sep 19 06:05:11.896519 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.896494 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" Sep 19 06:05:11.907384 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:11.906163 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn"] Sep 19 06:05:12.007420 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.007370 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25ea22eb-7a1e-453d-a971-081594589eea-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw\" (UID: \"25ea22eb-7a1e-453d-a971-081594589eea\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" Sep 19 06:05:12.007604 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.007448 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrq2w\" (UniqueName: \"kubernetes.io/projected/bd36f486-8ccc-4588-a3cb-32a9d818f9a2-kube-api-access-lrq2w\") pod \"payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn\" (UID: \"bd36f486-8ccc-4588-a3cb-32a9d818f9a2\") " pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" Sep 19 06:05:12.007604 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.007561 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6w9m\" (UniqueName: \"kubernetes.io/projected/25ea22eb-7a1e-453d-a971-081594589eea-kube-api-access-r6w9m\") pod \"payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw\" (UID: \"25ea22eb-7a1e-453d-a971-081594589eea\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" Sep 19 06:05:12.007604 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.007599 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd36f486-8ccc-4588-a3cb-32a9d818f9a2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn\" (UID: \"bd36f486-8ccc-4588-a3cb-32a9d818f9a2\") " pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" Sep 19 06:05:12.108970 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.108924 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6w9m\" (UniqueName: \"kubernetes.io/projected/25ea22eb-7a1e-453d-a971-081594589eea-kube-api-access-r6w9m\") pod \"payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw\" (UID: \"25ea22eb-7a1e-453d-a971-081594589eea\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" Sep 19 06:05:12.108970 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.108967 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd36f486-8ccc-4588-a3cb-32a9d818f9a2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn\" (UID: \"bd36f486-8ccc-4588-a3cb-32a9d818f9a2\") " pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" Sep 19 06:05:12.109248 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.109042 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25ea22eb-7a1e-453d-a971-081594589eea-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw\" (UID: \"25ea22eb-7a1e-453d-a971-081594589eea\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" Sep 19 06:05:12.109248 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.109060 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrq2w\" (UniqueName: \"kubernetes.io/projected/bd36f486-8ccc-4588-a3cb-32a9d818f9a2-kube-api-access-lrq2w\") pod \"payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn\" (UID: \"bd36f486-8ccc-4588-a3cb-32a9d818f9a2\") " pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" Sep 19 06:05:12.109799 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.109776 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25ea22eb-7a1e-453d-a971-081594589eea-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw\" (UID: \"25ea22eb-7a1e-453d-a971-081594589eea\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" Sep 19 06:05:12.109889 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.109821 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd36f486-8ccc-4588-a3cb-32a9d818f9a2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn\" (UID: \"bd36f486-8ccc-4588-a3cb-32a9d818f9a2\") " pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" Sep 19 06:05:12.118071 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.118045 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrq2w\" (UniqueName: \"kubernetes.io/projected/bd36f486-8ccc-4588-a3cb-32a9d818f9a2-kube-api-access-lrq2w\") pod \"payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn\" (UID: \"bd36f486-8ccc-4588-a3cb-32a9d818f9a2\") " pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" Sep 19 06:05:12.118287 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.118263 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6w9m\" (UniqueName: \"kubernetes.io/projected/25ea22eb-7a1e-453d-a971-081594589eea-kube-api-access-r6w9m\") pod \"payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw\" (UID: \"25ea22eb-7a1e-453d-a971-081594589eea\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" Sep 19 06:05:12.185992 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.185949 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" Sep 19 06:05:12.209433 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.209385 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" Sep 19 06:05:12.464354 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.464325 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw"] Sep 19 06:05:12.465706 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:05:12.465665 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25ea22eb_7a1e_453d_a971_081594589eea.slice/crio-6e4e9ca145beb4f490ab212d8b0a8e248a7454d345fb439af1a2f35e4a45fef6 WatchSource:0}: Error finding container 6e4e9ca145beb4f490ab212d8b0a8e248a7454d345fb439af1a2f35e4a45fef6: Status 404 returned error can't find the container with id 6e4e9ca145beb4f490ab212d8b0a8e248a7454d345fb439af1a2f35e4a45fef6 Sep 19 06:05:12.493026 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.493002 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn"] Sep 19 06:05:12.497514 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.497475 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" event={"ID":"25ea22eb-7a1e-453d-a971-081594589eea","Type":"ContainerStarted","Data":"6e4e9ca145beb4f490ab212d8b0a8e248a7454d345fb439af1a2f35e4a45fef6"} Sep 19 06:05:12.497662 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:05:12.497622 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd36f486_8ccc_4588_a3cb_32a9d818f9a2.slice/crio-27997873c81ef479792492cd25e9993372fb3ee16e8e775c3b532962e555475a WatchSource:0}: Error finding container 27997873c81ef479792492cd25e9993372fb3ee16e8e775c3b532962e555475a: Status 404 returned error can't find the container with id 27997873c81ef479792492cd25e9993372fb3ee16e8e775c3b532962e555475a Sep 19 06:05:12.711635 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:12.711600 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:13.506594 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:13.506554 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" event={"ID":"25ea22eb-7a1e-453d-a971-081594589eea","Type":"ContainerStarted","Data":"5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc"} Sep 19 06:05:13.507100 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:13.506629 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" Sep 19 06:05:13.508514 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:13.508484 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" event={"ID":"bd36f486-8ccc-4588-a3cb-32a9d818f9a2","Type":"ContainerStarted","Data":"9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e"} Sep 19 06:05:13.508668 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:13.508530 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" event={"ID":"bd36f486-8ccc-4588-a3cb-32a9d818f9a2","Type":"ContainerStarted","Data":"27997873c81ef479792492cd25e9993372fb3ee16e8e775c3b532962e555475a"} Sep 19 06:05:13.508668 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:13.508573 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" Sep 19 06:05:13.527213 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:13.527166 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" podStartSLOduration=2.527150457 podStartE2EDuration="2.527150457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:05:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:05:13.524386854 +0000 UTC m=+1441.517614389" watchObservedRunningTime="2026-09-19 06:05:13.527150457 +0000 UTC m=+1441.520377989" Sep 19 06:05:13.544177 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:13.544117 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" podStartSLOduration=2.5440959899999998 podStartE2EDuration="2.54409599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:05:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:05:13.540474818 +0000 UTC m=+1441.533702352" watchObservedRunningTime="2026-09-19 06:05:13.54409599 +0000 UTC m=+1441.537323526" Sep 19 06:05:13.544658 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:13.544605 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:14.529134 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:14.529100 2584 scope.go:117] "RemoveContainer" containerID="98c8df9a5f54a1520f011feb608f9983a57cf3ca213faa19952264b9454ab10f" Sep 19 06:05:24.518256 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:24.518221 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" Sep 19 06:05:24.518785 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:24.518303 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" Sep 19 06:05:30.389339 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:30.389305 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:30.450813 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:30.450782 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:34.010574 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:34.010533 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:34.798325 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:34.798291 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:35.453146 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:35.453109 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:36.205232 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:36.205201 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:36.989522 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:36.989485 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:37.501626 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:37.501584 2584 patch_prober.go:28] interesting pod/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 19 06:05:37.501813 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:37.501667 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 19 06:05:37.516231 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:37.516189 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:37.803302 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:37.803221 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:37.979154 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:37.979126 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:38.370780 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:38.370742 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:38.674909 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:38.674819 2584 generic.go:356] "Generic (PLEG): container finished" podID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerID="60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a" exitCode=0 Sep 19 06:05:38.675058 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:38.674904 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" event={"ID":"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514","Type":"ContainerDied","Data":"60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a"} Sep 19 06:05:38.675344 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:38.675329 2584 scope.go:117] "RemoveContainer" containerID="60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a" Sep 19 06:05:38.676608 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:38.676575 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:05:38.676742 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:38.676661 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:05:38.676742 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:38.676710 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:05:38.750561 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:38.750538 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:38.851050 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:38.851016 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:39.682493 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:39.682458 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" event={"ID":"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514","Type":"ContainerStarted","Data":"932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff"} Sep 19 06:05:40.533177 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:40.533136 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:40.533389 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:40.533368 2584 patch_prober.go:28] interesting pod/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body= Sep 19 06:05:40.533500 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:40.533445 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" Sep 19 06:05:41.305337 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:41.305301 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:41.533881 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:41.533844 2584 patch_prober.go:28] interesting pod/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body= Sep 19 06:05:41.534064 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:41.533899 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" Sep 19 06:05:41.913665 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:41.913621 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:42.008419 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:42.008360 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:42.383009 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:42.382976 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:42.533381 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:42.533348 2584 patch_prober.go:28] interesting pod/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body= Sep 19 06:05:42.533581 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:42.533422 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" Sep 19 06:05:43.288178 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:43.288141 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:43.534132 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:43.534097 2584 patch_prober.go:28] interesting pod/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body= Sep 19 06:05:43.534537 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:43.534168 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" Sep 19 06:05:44.372974 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:44.372939 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:44.533642 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:44.533598 2584 patch_prober.go:28] interesting pod/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body= Sep 19 06:05:44.533854 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:44.533693 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" Sep 19 06:05:45.115301 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:45.115269 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:45.533955 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:45.533868 2584 patch_prober.go:28] interesting pod/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body= Sep 19 06:05:45.533955 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:45.533938 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" Sep 19 06:05:46.537497 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:46.537467 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:46.537955 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:46.537756 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:46.538423 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:46.538390 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:05:46.611356 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:46.611319 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:46.711752 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:46.711720 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:47.263969 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.263927 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:47.360186 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.360153 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:47.702925 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.702897 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:47.802635 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.802600 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:47.839183 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.839143 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw"] Sep 19 06:05:47.843592 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.843572 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" Sep 19 06:05:47.845781 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.845758 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-aea238\"" Sep 19 06:05:47.859230 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.859197 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw"] Sep 19 06:05:47.899475 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.899433 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmvbt\" (UniqueName: \"kubernetes.io/projected/12023488-5f35-4f05-9d04-d0c1a8c3036b-kube-api-access-bmvbt\") pod \"payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw\" (UID: \"12023488-5f35-4f05-9d04-d0c1a8c3036b\") " pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" Sep 19 06:05:47.899681 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.899550 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12023488-5f35-4f05-9d04-d0c1a8c3036b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw\" (UID: \"12023488-5f35-4f05-9d04-d0c1a8c3036b\") " pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" Sep 19 06:05:47.899751 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.899700 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc"] Sep 19 06:05:47.903868 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.903850 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" Sep 19 06:05:47.906048 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.906024 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-aea238-dockercfg-5phmn\"" Sep 19 06:05:47.913184 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:47.913160 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc"] Sep 19 06:05:48.001042 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.000945 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/481fad5e-8e3e-43a4-a3c4-ff8176203686-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc\" (UID: \"481fad5e-8e3e-43a4-a3c4-ff8176203686\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" Sep 19 06:05:48.001042 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.001001 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrpv5\" (UniqueName: \"kubernetes.io/projected/481fad5e-8e3e-43a4-a3c4-ff8176203686-kube-api-access-rrpv5\") pod \"payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc\" (UID: \"481fad5e-8e3e-43a4-a3c4-ff8176203686\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" Sep 19 06:05:48.001281 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.001124 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmvbt\" (UniqueName: \"kubernetes.io/projected/12023488-5f35-4f05-9d04-d0c1a8c3036b-kube-api-access-bmvbt\") pod \"payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw\" (UID: \"12023488-5f35-4f05-9d04-d0c1a8c3036b\") " pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" Sep 19 06:05:48.001281 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.001218 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12023488-5f35-4f05-9d04-d0c1a8c3036b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw\" (UID: \"12023488-5f35-4f05-9d04-d0c1a8c3036b\") " pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" Sep 19 06:05:48.002138 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.002113 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12023488-5f35-4f05-9d04-d0c1a8c3036b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw\" (UID: \"12023488-5f35-4f05-9d04-d0c1a8c3036b\") " pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" Sep 19 06:05:48.008721 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.008693 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmvbt\" (UniqueName: \"kubernetes.io/projected/12023488-5f35-4f05-9d04-d0c1a8c3036b-kube-api-access-bmvbt\") pod \"payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw\" (UID: \"12023488-5f35-4f05-9d04-d0c1a8c3036b\") " pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" Sep 19 06:05:48.083201 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.083137 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:05:48.102482 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.102449 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/481fad5e-8e3e-43a4-a3c4-ff8176203686-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc\" (UID: \"481fad5e-8e3e-43a4-a3c4-ff8176203686\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" Sep 19 06:05:48.102662 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.102488 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrpv5\" (UniqueName: \"kubernetes.io/projected/481fad5e-8e3e-43a4-a3c4-ff8176203686-kube-api-access-rrpv5\") pod \"payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc\" (UID: \"481fad5e-8e3e-43a4-a3c4-ff8176203686\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" Sep 19 06:05:48.103177 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.103156 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/481fad5e-8e3e-43a4-a3c4-ff8176203686-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc\" (UID: \"481fad5e-8e3e-43a4-a3c4-ff8176203686\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" Sep 19 06:05:48.109585 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.109561 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrpv5\" (UniqueName: \"kubernetes.io/projected/481fad5e-8e3e-43a4-a3c4-ff8176203686-kube-api-access-rrpv5\") pod \"payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc\" (UID: \"481fad5e-8e3e-43a4-a3c4-ff8176203686\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" Sep 19 06:05:48.162162 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.162132 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" Sep 19 06:05:48.216249 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.216211 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" Sep 19 06:05:48.434861 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.434829 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw"] Sep 19 06:05:48.436949 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:05:48.436912 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12023488_5f35_4f05_9d04_d0c1a8c3036b.slice/crio-043c446c6fac2006c23a4c5913fb071b5f278b0cc82d6e46d330e5390621c75f WatchSource:0}: Error finding container 043c446c6fac2006c23a4c5913fb071b5f278b0cc82d6e46d330e5390621c75f: Status 404 returned error can't find the container with id 043c446c6fac2006c23a4c5913fb071b5f278b0cc82d6e46d330e5390621c75f Sep 19 06:05:48.507281 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.507248 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc"] Sep 19 06:05:48.508379 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:05:48.508358 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod481fad5e_8e3e_43a4_a3c4_ff8176203686.slice/crio-365c4b7f100ebb728640ab5a99a1e07ed29a3ad1fdd74752020d75e0c81df2b6 WatchSource:0}: Error finding container 365c4b7f100ebb728640ab5a99a1e07ed29a3ad1fdd74752020d75e0c81df2b6: Status 404 returned error can't find the container with id 365c4b7f100ebb728640ab5a99a1e07ed29a3ad1fdd74752020d75e0c81df2b6 Sep 19 06:05:48.754832 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.754793 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" event={"ID":"481fad5e-8e3e-43a4-a3c4-ff8176203686","Type":"ContainerStarted","Data":"eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1"} Sep 19 06:05:48.755272 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.754857 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" event={"ID":"481fad5e-8e3e-43a4-a3c4-ff8176203686","Type":"ContainerStarted","Data":"365c4b7f100ebb728640ab5a99a1e07ed29a3ad1fdd74752020d75e0c81df2b6"} Sep 19 06:05:48.755272 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.754886 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" Sep 19 06:05:48.756701 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.756674 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" event={"ID":"12023488-5f35-4f05-9d04-d0c1a8c3036b","Type":"ContainerStarted","Data":"42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0"} Sep 19 06:05:48.756826 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.756723 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" event={"ID":"12023488-5f35-4f05-9d04-d0c1a8c3036b","Type":"ContainerStarted","Data":"043c446c6fac2006c23a4c5913fb071b5f278b0cc82d6e46d330e5390621c75f"} Sep 19 06:05:48.756826 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.756750 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" Sep 19 06:05:48.770864 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.770799 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" podStartSLOduration=1.7707801029999999 podStartE2EDuration="1.770780103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:05:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:05:48.768495773 +0000 UTC m=+1476.761723302" watchObservedRunningTime="2026-09-19 06:05:48.770780103 +0000 UTC m=+1476.764007661" Sep 19 06:05:48.784921 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:48.784861 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" podStartSLOduration=1.784840552 podStartE2EDuration="1.784840552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:05:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:05:48.782304882 +0000 UTC m=+1476.775532414" watchObservedRunningTime="2026-09-19 06:05:48.784840552 +0000 UTC m=+1476.778068084" Sep 19 06:05:59.768122 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:59.768089 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" Sep 19 06:05:59.768642 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:05:59.768159 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" Sep 19 06:06:09.687742 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:09.687688 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:10.309777 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:10.309739 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:11.187539 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:11.187498 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:11.816684 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:11.816646 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:11.917300 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:11.917266 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:12.484881 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:12.484844 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:13.193142 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:13.193107 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:13.818363 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:13.818335 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:14.341121 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:14.341095 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 06:06:14.347093 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:14.347069 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 06:06:14.556753 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:14.556720 2584 scope.go:117] "RemoveContainer" containerID="6d0b10c01c3d63787d4048fa918b1d5df7c523550db0239764fc7ab01fc1eb65" Sep 19 06:06:14.566583 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:14.566558 2584 scope.go:117] "RemoveContainer" containerID="dd5eaec97724cbc86c92f70dfb3a3420d2f792a36cad7f78422d685535bcc4d6" Sep 19 06:06:14.594207 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:14.594147 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:14.862542 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:14.862453 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52"] Sep 19 06:06:14.867508 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:14.867484 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52" Sep 19 06:06:14.870148 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:14.870116 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:06:14.873097 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:14.873068 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52"] Sep 19 06:06:14.889935 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:14.889904 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf6ab88-452d-4928-805d-5d718e65fb3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-311e2d-jwb52\" (UID: \"3cf6ab88-452d-4928-805d-5d718e65fb3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52" Sep 19 06:06:14.991493 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:14.991447 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf6ab88-452d-4928-805d-5d718e65fb3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-311e2d-jwb52\" (UID: \"3cf6ab88-452d-4928-805d-5d718e65fb3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52" Sep 19 06:06:14.992082 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:14.992063 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf6ab88-452d-4928-805d-5d718e65fb3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-311e2d-jwb52\" (UID: \"3cf6ab88-452d-4928-805d-5d718e65fb3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52" Sep 19 06:06:15.182211 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:15.182122 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52" Sep 19 06:06:15.395786 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:15.395752 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:15.454679 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:15.454647 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52"] Sep 19 06:06:15.457368 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:06:15.457338 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf6ab88_452d_4928_805d_5d718e65fb3f.slice/crio-7c7f0c8ab0c45bcdc720077aa7b71b8ae6b285a068d2f40f65b3bd300905ca40 WatchSource:0}: Error finding container 7c7f0c8ab0c45bcdc720077aa7b71b8ae6b285a068d2f40f65b3bd300905ca40: Status 404 returned error can't find the container with id 7c7f0c8ab0c45bcdc720077aa7b71b8ae6b285a068d2f40f65b3bd300905ca40 Sep 19 06:06:15.926727 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:15.926683 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52" event={"ID":"3cf6ab88-452d-4928-805d-5d718e65fb3f","Type":"ContainerStarted","Data":"947a4af26b3e6e993628f57743fcaa6d0ab3454b519722e80bf4aeaea4a846da"} Sep 19 06:06:15.927150 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:15.926750 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52" event={"ID":"3cf6ab88-452d-4928-805d-5d718e65fb3f","Type":"ContainerStarted","Data":"7c7f0c8ab0c45bcdc720077aa7b71b8ae6b285a068d2f40f65b3bd300905ca40"} Sep 19 06:06:15.942631 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:15.942569 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52" podStartSLOduration=1.9425438659999998 podStartE2EDuration="1.942543866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:06:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:06:15.940231377 +0000 UTC m=+1503.933458899" watchObservedRunningTime="2026-09-19 06:06:15.942543866 +0000 UTC m=+1503.935771400" Sep 19 06:06:16.092963 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:16.092932 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:16.793722 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:16.793689 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:16.934790 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:16.934749 2584 generic.go:356] "Generic (PLEG): container finished" podID="3cf6ab88-452d-4928-805d-5d718e65fb3f" containerID="947a4af26b3e6e993628f57743fcaa6d0ab3454b519722e80bf4aeaea4a846da" exitCode=0 Sep 19 06:06:16.935217 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:16.934840 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52" event={"ID":"3cf6ab88-452d-4928-805d-5d718e65fb3f","Type":"ContainerDied","Data":"947a4af26b3e6e993628f57743fcaa6d0ab3454b519722e80bf4aeaea4a846da"} Sep 19 06:06:17.517334 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:17.517301 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:18.087137 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:18.087106 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52" Sep 19 06:06:18.122167 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:18.122130 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf6ab88-452d-4928-805d-5d718e65fb3f-maas-api-service-ca\") pod \"3cf6ab88-452d-4928-805d-5d718e65fb3f\" (UID: \"3cf6ab88-452d-4928-805d-5d718e65fb3f\") " Sep 19 06:06:18.122508 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:18.122485 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf6ab88-452d-4928-805d-5d718e65fb3f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3cf6ab88-452d-4928-805d-5d718e65fb3f" (UID: "3cf6ab88-452d-4928-805d-5d718e65fb3f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:06:18.223745 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:18.223641 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf6ab88-452d-4928-805d-5d718e65fb3f-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:18.381673 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:18.381638 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:18.948058 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:18.948019 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52" Sep 19 06:06:18.948237 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:18.948050 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52" event={"ID":"3cf6ab88-452d-4928-805d-5d718e65fb3f","Type":"ContainerDied","Data":"7c7f0c8ab0c45bcdc720077aa7b71b8ae6b285a068d2f40f65b3bd300905ca40"} Sep 19 06:06:18.948237 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:18.948103 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c7f0c8ab0c45bcdc720077aa7b71b8ae6b285a068d2f40f65b3bd300905ca40" Sep 19 06:06:19.004636 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:19.004606 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:19.674156 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:19.674122 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:20.411001 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:20.410966 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:24.986676 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:24.986642 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:30.545129 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.545094 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:30.560939 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.560907 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn"] Sep 19 06:06:30.561137 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.561111 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" podUID="bd36f486-8ccc-4588-a3cb-32a9d818f9a2" containerName="payload-processing" containerID="cri-o://9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e" gracePeriod=30 Sep 19 06:06:30.579133 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.579100 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw"] Sep 19 06:06:30.579607 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.579420 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" podUID="25ea22eb-7a1e-453d-a971-081594589eea" containerName="payload-pre-processing" containerID="cri-o://5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc" gracePeriod=30 Sep 19 06:06:30.602914 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.602878 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:30.658700 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.658676 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:30.729511 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.729452 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52"] Sep 19 06:06:30.730482 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.730418 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:30.732536 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.732501 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-311e2d-jwb52"] Sep 19 06:06:30.787040 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.787005 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:30.864780 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.864751 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" Sep 19 06:06:30.868088 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.868060 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" Sep 19 06:06:30.961650 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.961602 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrq2w\" (UniqueName: \"kubernetes.io/projected/bd36f486-8ccc-4588-a3cb-32a9d818f9a2-kube-api-access-lrq2w\") pod \"bd36f486-8ccc-4588-a3cb-32a9d818f9a2\" (UID: \"bd36f486-8ccc-4588-a3cb-32a9d818f9a2\") " Sep 19 06:06:30.961831 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.961775 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd36f486-8ccc-4588-a3cb-32a9d818f9a2-plugins-config-volume\") pod \"bd36f486-8ccc-4588-a3cb-32a9d818f9a2\" (UID: \"bd36f486-8ccc-4588-a3cb-32a9d818f9a2\") " Sep 19 06:06:30.961831 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.961820 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25ea22eb-7a1e-453d-a971-081594589eea-plugins-config-volume\") pod \"25ea22eb-7a1e-453d-a971-081594589eea\" (UID: \"25ea22eb-7a1e-453d-a971-081594589eea\") " Sep 19 06:06:30.961956 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.961866 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6w9m\" (UniqueName: \"kubernetes.io/projected/25ea22eb-7a1e-453d-a971-081594589eea-kube-api-access-r6w9m\") pod \"25ea22eb-7a1e-453d-a971-081594589eea\" (UID: \"25ea22eb-7a1e-453d-a971-081594589eea\") " Sep 19 06:06:30.962254 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.962207 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd36f486-8ccc-4588-a3cb-32a9d818f9a2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bd36f486-8ccc-4588-a3cb-32a9d818f9a2" (UID: "bd36f486-8ccc-4588-a3cb-32a9d818f9a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:06:30.962363 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.962243 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25ea22eb-7a1e-453d-a971-081594589eea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "25ea22eb-7a1e-453d-a971-081594589eea" (UID: "25ea22eb-7a1e-453d-a971-081594589eea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:06:30.962755 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.962731 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd36f486-8ccc-4588-a3cb-32a9d818f9a2-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:30.962883 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.962760 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/25ea22eb-7a1e-453d-a971-081594589eea-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:30.964131 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.964105 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd36f486-8ccc-4588-a3cb-32a9d818f9a2-kube-api-access-lrq2w" (OuterVolumeSpecName: "kube-api-access-lrq2w") pod "bd36f486-8ccc-4588-a3cb-32a9d818f9a2" (UID: "bd36f486-8ccc-4588-a3cb-32a9d818f9a2"). InnerVolumeSpecName "kube-api-access-lrq2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:06:30.964467 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:30.964448 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25ea22eb-7a1e-453d-a971-081594589eea-kube-api-access-r6w9m" (OuterVolumeSpecName: "kube-api-access-r6w9m") pod "25ea22eb-7a1e-453d-a971-081594589eea" (UID: "25ea22eb-7a1e-453d-a971-081594589eea"). InnerVolumeSpecName "kube-api-access-r6w9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:06:31.034537 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.034499 2584 generic.go:356] "Generic (PLEG): container finished" podID="25ea22eb-7a1e-453d-a971-081594589eea" containerID="5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc" exitCode=0 Sep 19 06:06:31.034741 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.034556 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" Sep 19 06:06:31.034741 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.034700 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" event={"ID":"25ea22eb-7a1e-453d-a971-081594589eea","Type":"ContainerDied","Data":"5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc"} Sep 19 06:06:31.034874 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.034742 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw" event={"ID":"25ea22eb-7a1e-453d-a971-081594589eea","Type":"ContainerDied","Data":"6e4e9ca145beb4f490ab212d8b0a8e248a7454d345fb439af1a2f35e4a45fef6"} Sep 19 06:06:31.034874 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.034769 2584 scope.go:117] "RemoveContainer" containerID="5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc" Sep 19 06:06:31.038716 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.038681 2584 generic.go:356] "Generic (PLEG): container finished" podID="bd36f486-8ccc-4588-a3cb-32a9d818f9a2" containerID="9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e" exitCode=0 Sep 19 06:06:31.038874 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.038743 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" event={"ID":"bd36f486-8ccc-4588-a3cb-32a9d818f9a2","Type":"ContainerDied","Data":"9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e"} Sep 19 06:06:31.038874 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.038790 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" event={"ID":"bd36f486-8ccc-4588-a3cb-32a9d818f9a2","Type":"ContainerDied","Data":"27997873c81ef479792492cd25e9993372fb3ee16e8e775c3b532962e555475a"} Sep 19 06:06:31.038874 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.038868 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn" Sep 19 06:06:31.055703 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.055654 2584 scope.go:117] "RemoveContainer" containerID="5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc" Sep 19 06:06:31.056987 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:06:31.056953 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc\": container with ID starting with 5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc not found: ID does not exist" containerID="5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc" Sep 19 06:06:31.057155 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.056997 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc"} err="failed to get container status \"5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc\": rpc error: code = NotFound desc = could not find container \"5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc\": container with ID starting with 5b7217968ec1a42d5b01940da2a9f67d65a15c54197a9cce2cd694f27a0f87dc not found: ID does not exist" Sep 19 06:06:31.057155 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.057024 2584 scope.go:117] "RemoveContainer" containerID="9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e" Sep 19 06:06:31.064779 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.064750 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6w9m\" (UniqueName: \"kubernetes.io/projected/25ea22eb-7a1e-453d-a971-081594589eea-kube-api-access-r6w9m\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:31.064898 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.064782 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrq2w\" (UniqueName: \"kubernetes.io/projected/bd36f486-8ccc-4588-a3cb-32a9d818f9a2-kube-api-access-lrq2w\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:31.071784 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.071760 2584 scope.go:117] "RemoveContainer" containerID="9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e" Sep 19 06:06:31.072164 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:06:31.072142 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e\": container with ID starting with 9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e not found: ID does not exist" containerID="9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e" Sep 19 06:06:31.072244 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.072172 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e"} err="failed to get container status \"9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e\": rpc error: code = NotFound desc = could not find container \"9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e\": container with ID starting with 9cd9e3ce52d7f1aecb084e5d191c344d2be713962aa755e29e1cd55ae685829e not found: ID does not exist" Sep 19 06:06:31.072455 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.072436 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn"] Sep 19 06:06:31.078136 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.078104 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-311e2d-9d99bf579-4nccn"] Sep 19 06:06:31.086881 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.086801 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw"] Sep 19 06:06:31.093633 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.093603 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-311e2d-79dcd759c4-6jttw"] Sep 19 06:06:31.124621 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.124593 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:31.224779 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.224744 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:31.892979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:31.892890 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:32.107063 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:32.107030 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:32.207641 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:32.207546 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:32.267866 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:32.267835 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25ea22eb-7a1e-453d-a971-081594589eea" path="/var/lib/kubelet/pods/25ea22eb-7a1e-453d-a971-081594589eea/volumes" Sep 19 06:06:32.268198 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:32.268185 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cf6ab88-452d-4928-805d-5d718e65fb3f" path="/var/lib/kubelet/pods/3cf6ab88-452d-4928-805d-5d718e65fb3f/volumes" Sep 19 06:06:32.268552 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:32.268538 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd36f486-8ccc-4588-a3cb-32a9d818f9a2" path="/var/lib/kubelet/pods/bd36f486-8ccc-4588-a3cb-32a9d818f9a2/volumes" Sep 19 06:06:32.574091 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:32.574000 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:33.221583 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:33.221552 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:34.069088 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:34.069049 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:34.685239 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:34.685209 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:35.455664 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:35.455631 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:36.918819 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:36.918792 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:37.021374 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.021342 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:37.183559 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.183473 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx"] Sep 19 06:06:37.183776 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.183742 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" containerID="cri-o://932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff" gracePeriod=30 Sep 19 06:06:37.202885 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.202857 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:37.515714 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.515631 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp"] Sep 19 06:06:37.516125 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.516113 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25ea22eb-7a1e-453d-a971-081594589eea" containerName="payload-pre-processing" Sep 19 06:06:37.516171 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.516127 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="25ea22eb-7a1e-453d-a971-081594589eea" containerName="payload-pre-processing" Sep 19 06:06:37.516171 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.516143 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd36f486-8ccc-4588-a3cb-32a9d818f9a2" containerName="payload-processing" Sep 19 06:06:37.516171 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.516148 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd36f486-8ccc-4588-a3cb-32a9d818f9a2" containerName="payload-processing" Sep 19 06:06:37.516171 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.516162 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf6ab88-452d-4928-805d-5d718e65fb3f" containerName="revoke-keys" Sep 19 06:06:37.516171 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.516167 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf6ab88-452d-4928-805d-5d718e65fb3f" containerName="revoke-keys" Sep 19 06:06:37.516357 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.516244 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf6ab88-452d-4928-805d-5d718e65fb3f" containerName="revoke-keys" Sep 19 06:06:37.516357 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.516254 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd36f486-8ccc-4588-a3cb-32a9d818f9a2" containerName="payload-processing" Sep 19 06:06:37.516357 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.516262 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="25ea22eb-7a1e-453d-a971-081594589eea" containerName="payload-pre-processing" Sep 19 06:06:37.519522 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.519504 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp" Sep 19 06:06:37.521645 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.521623 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:06:37.528668 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.528643 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp"] Sep 19 06:06:37.632582 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.632539 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4015f99c-91c8-4885-9eab-585c4ec38607-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-aea238-bh7xp\" (UID: \"4015f99c-91c8-4885-9eab-585c4ec38607\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp" Sep 19 06:06:37.734197 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.734162 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4015f99c-91c8-4885-9eab-585c4ec38607-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-aea238-bh7xp\" (UID: \"4015f99c-91c8-4885-9eab-585c4ec38607\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp" Sep 19 06:06:37.734827 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.734807 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4015f99c-91c8-4885-9eab-585c4ec38607-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-aea238-bh7xp\" (UID: \"4015f99c-91c8-4885-9eab-585c4ec38607\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp" Sep 19 06:06:37.778287 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.778186 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:37.836734 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:37.836703 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp" Sep 19 06:06:38.111060 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:38.111036 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp"] Sep 19 06:06:38.113204 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:06:38.113175 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4015f99c_91c8_4885_9eab_585c4ec38607.slice/crio-093d034ba29cee4f66fdf102f3378d11a5fe997f46587fe43851255fb57054bb WatchSource:0}: Error finding container 093d034ba29cee4f66fdf102f3378d11a5fe997f46587fe43851255fb57054bb: Status 404 returned error can't find the container with id 093d034ba29cee4f66fdf102f3378d11a5fe997f46587fe43851255fb57054bb Sep 19 06:06:38.486421 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:38.486366 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:39.092046 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:39.092013 2584 generic.go:356] "Generic (PLEG): container finished" podID="4015f99c-91c8-4885-9eab-585c4ec38607" containerID="0ebce582c6bc97d4daa64e02f2b6639d3687a6b037d908662302ff509fa09df1" exitCode=0 Sep 19 06:06:39.092213 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:39.092101 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp" event={"ID":"4015f99c-91c8-4885-9eab-585c4ec38607","Type":"ContainerDied","Data":"0ebce582c6bc97d4daa64e02f2b6639d3687a6b037d908662302ff509fa09df1"} Sep 19 06:06:39.092213 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:39.092152 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp" event={"ID":"4015f99c-91c8-4885-9eab-585c4ec38607","Type":"ContainerStarted","Data":"093d034ba29cee4f66fdf102f3378d11a5fe997f46587fe43851255fb57054bb"} Sep 19 06:06:39.226730 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:39.226684 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:39.927337 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:39.927294 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:40.236812 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:40.236788 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp" Sep 19 06:06:40.363176 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:40.363143 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4015f99c-91c8-4885-9eab-585c4ec38607-maas-api-service-ca\") pod \"4015f99c-91c8-4885-9eab-585c4ec38607\" (UID: \"4015f99c-91c8-4885-9eab-585c4ec38607\") " Sep 19 06:06:40.363600 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:40.363574 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4015f99c-91c8-4885-9eab-585c4ec38607-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4015f99c-91c8-4885-9eab-585c4ec38607" (UID: "4015f99c-91c8-4885-9eab-585c4ec38607"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:06:40.464765 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:40.464685 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4015f99c-91c8-4885-9eab-585c4ec38607-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:40.674706 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:40.674629 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:41.107764 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:41.107728 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp" event={"ID":"4015f99c-91c8-4885-9eab-585c4ec38607","Type":"ContainerDied","Data":"093d034ba29cee4f66fdf102f3378d11a5fe997f46587fe43851255fb57054bb"} Sep 19 06:06:41.107945 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:41.107769 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp" Sep 19 06:06:41.107945 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:41.107775 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="093d034ba29cee4f66fdf102f3378d11a5fe997f46587fe43851255fb57054bb" Sep 19 06:06:41.313820 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:41.313778 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:41.981494 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:41.981466 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:42.447384 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.447359 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:06:42.585708 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.585679 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-data\") pod \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " Sep 19 06:06:42.585872 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.585743 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-envoy\") pod \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " Sep 19 06:06:42.585872 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.585807 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-podinfo\") pod \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " Sep 19 06:06:42.586006 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.585870 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmfph\" (UniqueName: \"kubernetes.io/projected/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-kube-api-access-qmfph\") pod \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " Sep 19 06:06:42.586006 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.585926 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-credential-socket\") pod \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " Sep 19 06:06:42.586006 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.585942 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-data" (OuterVolumeSpecName: "istio-data") pod "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" (UID: "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:06:42.586006 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.585971 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-workload-socket\") pod \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " Sep 19 06:06:42.586226 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.586012 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-token\") pod \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " Sep 19 06:06:42.586226 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.586047 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-workload-certs\") pod \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " Sep 19 06:06:42.586226 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.586074 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istiod-ca-cert\") pod \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\" (UID: \"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514\") " Sep 19 06:06:42.586226 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.586190 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" (UID: "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:06:42.586487 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.586226 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" (UID: "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:06:42.586487 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.586347 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" (UID: "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:06:42.586602 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.586545 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-credential-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:42.586602 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.586567 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-workload-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:42.586602 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.586582 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-workload-certs\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:42.586602 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.586584 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" (UID: "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:06:42.586602 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.586596 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-data\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:42.588537 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.588502 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" (UID: "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 06:06:42.588636 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.588601 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-kube-api-access-qmfph" (OuterVolumeSpecName: "kube-api-access-qmfph") pod "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" (UID: "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514"). InnerVolumeSpecName "kube-api-access-qmfph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:06:42.588636 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.588612 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" (UID: "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:06:42.588636 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.588620 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-token" (OuterVolumeSpecName: "istio-token") pod "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" (UID: "1cc2b311-d4a2-4cd8-97f6-b6f8fc147514"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:06:42.621385 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.621356 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:42.688215 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.688179 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-token\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:42.688215 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.688211 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istiod-ca-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:42.688215 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.688222 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-envoy\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:42.688579 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.688232 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-istio-podinfo\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:42.688579 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:42.688240 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmfph\" (UniqueName: \"kubernetes.io/projected/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514-kube-api-access-qmfph\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:43.123774 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:43.123733 2584 generic.go:356] "Generic (PLEG): container finished" podID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerID="932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff" exitCode=0 Sep 19 06:06:43.123944 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:43.123789 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" Sep 19 06:06:43.123944 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:43.123819 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" event={"ID":"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514","Type":"ContainerDied","Data":"932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff"} Sep 19 06:06:43.123944 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:43.123872 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx" event={"ID":"1cc2b311-d4a2-4cd8-97f6-b6f8fc147514","Type":"ContainerDied","Data":"9279ff376c7a9be44f4eb52474beffd1e4615bc1766fcc90371b30cbcad633c3"} Sep 19 06:06:43.123944 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:43.123896 2584 scope.go:117] "RemoveContainer" containerID="932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff" Sep 19 06:06:43.134302 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:43.134282 2584 scope.go:117] "RemoveContainer" containerID="60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a" Sep 19 06:06:43.143225 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:43.143198 2584 scope.go:117] "RemoveContainer" containerID="932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff" Sep 19 06:06:43.143596 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:06:43.143557 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff\": container with ID starting with 932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff not found: ID does not exist" containerID="932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff" Sep 19 06:06:43.143718 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:43.143604 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff"} err="failed to get container status \"932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff\": rpc error: code = NotFound desc = could not find container \"932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff\": container with ID starting with 932d24751418f48007ac9b3143cb3081c211146ce7954bf42e31f377aa8931ff not found: ID does not exist" Sep 19 06:06:43.143718 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:43.143622 2584 scope.go:117] "RemoveContainer" containerID="60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a" Sep 19 06:06:43.143939 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:06:43.143915 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a\": container with ID starting with 60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a not found: ID does not exist" containerID="60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a" Sep 19 06:06:43.144006 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:43.143937 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a"} err="failed to get container status \"60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a\": rpc error: code = NotFound desc = could not find container \"60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a\": container with ID starting with 60973bef1a214bee165a5a54a5ded605b5d32dbed926e84aee67caef5e87f62a not found: ID does not exist" Sep 19 06:06:43.146326 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:43.146281 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx"] Sep 19 06:06:43.149256 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:43.149236 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-311e2d-openshift-default-57cc74d57b-7gztx"] Sep 19 06:06:44.267852 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:44.267821 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" path="/var/lib/kubelet/pods/1cc2b311-d4a2-4cd8-97f6-b6f8fc147514/volumes" Sep 19 06:06:51.548445 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:51.548383 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:53.127283 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.127253 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:53.151772 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.151743 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw"] Sep 19 06:06:53.152002 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.151974 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" podUID="12023488-5f35-4f05-9d04-d0c1a8c3036b" containerName="payload-processing" containerID="cri-o://42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0" gracePeriod=30 Sep 19 06:06:53.157883 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.157854 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc"] Sep 19 06:06:53.158087 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.158052 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" podUID="481fad5e-8e3e-43a4-a3c4-ff8176203686" containerName="payload-pre-processing" containerID="cri-o://eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1" gracePeriod=30 Sep 19 06:06:53.194674 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.194644 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:53.245866 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.245835 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:53.309828 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.309787 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp"] Sep 19 06:06:53.310493 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.310462 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:53.311829 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.311803 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-aea238-bh7xp"] Sep 19 06:06:53.369961 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.369910 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:53.428908 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.428880 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" Sep 19 06:06:53.459299 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.459276 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" Sep 19 06:06:53.495274 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.495241 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmvbt\" (UniqueName: \"kubernetes.io/projected/12023488-5f35-4f05-9d04-d0c1a8c3036b-kube-api-access-bmvbt\") pod \"12023488-5f35-4f05-9d04-d0c1a8c3036b\" (UID: \"12023488-5f35-4f05-9d04-d0c1a8c3036b\") " Sep 19 06:06:53.495467 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.495345 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrpv5\" (UniqueName: \"kubernetes.io/projected/481fad5e-8e3e-43a4-a3c4-ff8176203686-kube-api-access-rrpv5\") pod \"481fad5e-8e3e-43a4-a3c4-ff8176203686\" (UID: \"481fad5e-8e3e-43a4-a3c4-ff8176203686\") " Sep 19 06:06:53.495467 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.495368 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12023488-5f35-4f05-9d04-d0c1a8c3036b-plugins-config-volume\") pod \"12023488-5f35-4f05-9d04-d0c1a8c3036b\" (UID: \"12023488-5f35-4f05-9d04-d0c1a8c3036b\") " Sep 19 06:06:53.495467 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.495443 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/481fad5e-8e3e-43a4-a3c4-ff8176203686-plugins-config-volume\") pod \"481fad5e-8e3e-43a4-a3c4-ff8176203686\" (UID: \"481fad5e-8e3e-43a4-a3c4-ff8176203686\") " Sep 19 06:06:53.495938 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.495903 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12023488-5f35-4f05-9d04-d0c1a8c3036b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12023488-5f35-4f05-9d04-d0c1a8c3036b" (UID: "12023488-5f35-4f05-9d04-d0c1a8c3036b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:06:53.496047 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.495962 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/481fad5e-8e3e-43a4-a3c4-ff8176203686-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "481fad5e-8e3e-43a4-a3c4-ff8176203686" (UID: "481fad5e-8e3e-43a4-a3c4-ff8176203686"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:06:53.497544 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.497508 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12023488-5f35-4f05-9d04-d0c1a8c3036b-kube-api-access-bmvbt" (OuterVolumeSpecName: "kube-api-access-bmvbt") pod "12023488-5f35-4f05-9d04-d0c1a8c3036b" (UID: "12023488-5f35-4f05-9d04-d0c1a8c3036b"). InnerVolumeSpecName "kube-api-access-bmvbt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:06:53.497650 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.497626 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/481fad5e-8e3e-43a4-a3c4-ff8176203686-kube-api-access-rrpv5" (OuterVolumeSpecName: "kube-api-access-rrpv5") pod "481fad5e-8e3e-43a4-a3c4-ff8176203686" (UID: "481fad5e-8e3e-43a4-a3c4-ff8176203686"). InnerVolumeSpecName "kube-api-access-rrpv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:06:53.596647 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.596601 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrpv5\" (UniqueName: \"kubernetes.io/projected/481fad5e-8e3e-43a4-a3c4-ff8176203686-kube-api-access-rrpv5\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:53.596647 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.596639 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12023488-5f35-4f05-9d04-d0c1a8c3036b-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:53.596647 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.596648 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/481fad5e-8e3e-43a4-a3c4-ff8176203686-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:53.596647 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.596658 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmvbt\" (UniqueName: \"kubernetes.io/projected/12023488-5f35-4f05-9d04-d0c1a8c3036b-kube-api-access-bmvbt\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:06:53.673454 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.673357 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:53.774766 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:53.774735 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:54.192559 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.192522 2584 generic.go:356] "Generic (PLEG): container finished" podID="12023488-5f35-4f05-9d04-d0c1a8c3036b" containerID="42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0" exitCode=0 Sep 19 06:06:54.193040 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.192594 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" Sep 19 06:06:54.193040 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.192630 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" event={"ID":"12023488-5f35-4f05-9d04-d0c1a8c3036b","Type":"ContainerDied","Data":"42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0"} Sep 19 06:06:54.193040 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.192721 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw" event={"ID":"12023488-5f35-4f05-9d04-d0c1a8c3036b","Type":"ContainerDied","Data":"043c446c6fac2006c23a4c5913fb071b5f278b0cc82d6e46d330e5390621c75f"} Sep 19 06:06:54.193040 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.192760 2584 scope.go:117] "RemoveContainer" containerID="42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0" Sep 19 06:06:54.194304 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.194281 2584 generic.go:356] "Generic (PLEG): container finished" podID="481fad5e-8e3e-43a4-a3c4-ff8176203686" containerID="eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1" exitCode=0 Sep 19 06:06:54.194438 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.194339 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" event={"ID":"481fad5e-8e3e-43a4-a3c4-ff8176203686","Type":"ContainerDied","Data":"eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1"} Sep 19 06:06:54.194438 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.194345 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" Sep 19 06:06:54.194438 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.194381 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc" event={"ID":"481fad5e-8e3e-43a4-a3c4-ff8176203686","Type":"ContainerDied","Data":"365c4b7f100ebb728640ab5a99a1e07ed29a3ad1fdd74752020d75e0c81df2b6"} Sep 19 06:06:54.203661 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.203623 2584 scope.go:117] "RemoveContainer" containerID="42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0" Sep 19 06:06:54.204062 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:06:54.204036 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0\": container with ID starting with 42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0 not found: ID does not exist" containerID="42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0" Sep 19 06:06:54.204150 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.204072 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0"} err="failed to get container status \"42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0\": rpc error: code = NotFound desc = could not find container \"42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0\": container with ID starting with 42bed99462aeddfaf55046e00407408f304d01f876ed05e062767f87cc8be3b0 not found: ID does not exist" Sep 19 06:06:54.204150 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.204094 2584 scope.go:117] "RemoveContainer" containerID="eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1" Sep 19 06:06:54.213541 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.213524 2584 scope.go:117] "RemoveContainer" containerID="eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1" Sep 19 06:06:54.213860 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:06:54.213841 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1\": container with ID starting with eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1 not found: ID does not exist" containerID="eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1" Sep 19 06:06:54.213916 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.213869 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1"} err="failed to get container status \"eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1\": rpc error: code = NotFound desc = could not find container \"eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1\": container with ID starting with eb212cd19ec40839fe6597f87a9a4174c70a200c6cbfec3c8a077ddcb6d59ff1 not found: ID does not exist" Sep 19 06:06:54.223021 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.222996 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw"] Sep 19 06:06:54.228284 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.228258 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-aea238-84846b6d9c-m4hjw"] Sep 19 06:06:54.237847 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.237819 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc"] Sep 19 06:06:54.241742 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.241714 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-aea238-856f85988b-vj2bc"] Sep 19 06:06:54.268823 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.268793 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12023488-5f35-4f05-9d04-d0c1a8c3036b" path="/var/lib/kubelet/pods/12023488-5f35-4f05-9d04-d0c1a8c3036b/volumes" Sep 19 06:06:54.269156 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.269143 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4015f99c-91c8-4885-9eab-585c4ec38607" path="/var/lib/kubelet/pods/4015f99c-91c8-4885-9eab-585c4ec38607/volumes" Sep 19 06:06:54.269502 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.269489 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="481fad5e-8e3e-43a4-a3c4-ff8176203686" path="/var/lib/kubelet/pods/481fad5e-8e3e-43a4-a3c4-ff8176203686/volumes" Sep 19 06:06:54.383866 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:54.383833 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:55.011923 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:55.011891 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:55.788679 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:55.788650 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:56.452861 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:56.452822 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:57.085542 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:57.085508 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:59.375636 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.375602 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:59.475805 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.475768 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:59.662180 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.662097 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:06:59.983146 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983064 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95"] Sep 19 06:06:59.983597 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983583 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12023488-5f35-4f05-9d04-d0c1a8c3036b" containerName="payload-processing" Sep 19 06:06:59.983649 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983599 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="12023488-5f35-4f05-9d04-d0c1a8c3036b" containerName="payload-processing" Sep 19 06:06:59.983649 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983620 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" Sep 19 06:06:59.983649 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983626 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" Sep 19 06:06:59.983649 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983634 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4015f99c-91c8-4885-9eab-585c4ec38607" containerName="revoke-keys" Sep 19 06:06:59.983649 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983640 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4015f99c-91c8-4885-9eab-585c4ec38607" containerName="revoke-keys" Sep 19 06:06:59.983800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983655 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" Sep 19 06:06:59.983800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983661 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" Sep 19 06:06:59.983800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983668 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="481fad5e-8e3e-43a4-a3c4-ff8176203686" containerName="payload-pre-processing" Sep 19 06:06:59.983800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983673 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="481fad5e-8e3e-43a4-a3c4-ff8176203686" containerName="payload-pre-processing" Sep 19 06:06:59.983800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983735 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" Sep 19 06:06:59.983800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983744 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="12023488-5f35-4f05-9d04-d0c1a8c3036b" containerName="payload-processing" Sep 19 06:06:59.983800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983750 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="481fad5e-8e3e-43a4-a3c4-ff8176203686" containerName="payload-pre-processing" Sep 19 06:06:59.983800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983757 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4015f99c-91c8-4885-9eab-585c4ec38607" containerName="revoke-keys" Sep 19 06:06:59.983800 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.983764 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cc2b311-d4a2-4cd8-97f6-b6f8fc147514" containerName="istio-proxy" Sep 19 06:06:59.988285 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.988264 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95" Sep 19 06:06:59.990812 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.990791 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:06:59.997076 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:06:59.997051 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95"] Sep 19 06:07:00.060685 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:00.060650 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95\" (UID: \"0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95" Sep 19 06:07:00.161922 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:00.161882 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95\" (UID: \"0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95" Sep 19 06:07:00.162613 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:00.162587 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95\" (UID: \"0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95" Sep 19 06:07:00.255547 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:00.255464 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:00.300659 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:00.300626 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95" Sep 19 06:07:00.575495 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:00.575463 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95"] Sep 19 06:07:00.578556 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:07:00.578520 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa8ce47_0199_4231_8bbb_f2e6d81ed4c8.slice/crio-898d4b3a2b214a136bb16c21ca1d6818648a42f9c26d23ba2bdedb59d8b5e632 WatchSource:0}: Error finding container 898d4b3a2b214a136bb16c21ca1d6818648a42f9c26d23ba2bdedb59d8b5e632: Status 404 returned error can't find the container with id 898d4b3a2b214a136bb16c21ca1d6818648a42f9c26d23ba2bdedb59d8b5e632 Sep 19 06:07:00.986091 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:00.986055 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:01.240825 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:01.240735 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95" event={"ID":"0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8","Type":"ContainerStarted","Data":"69b919ad0986c7ed56aa4b68da2ee0550a176ebc6532eba6e8c9d5bcf966727e"} Sep 19 06:07:01.240825 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:01.240787 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95" event={"ID":"0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8","Type":"ContainerStarted","Data":"898d4b3a2b214a136bb16c21ca1d6818648a42f9c26d23ba2bdedb59d8b5e632"} Sep 19 06:07:01.255714 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:01.255662 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95" podStartSLOduration=2.255645351 podStartE2EDuration="2.255645351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:06:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:07:01.253738479 +0000 UTC m=+1549.246966005" watchObservedRunningTime="2026-09-19 06:07:01.255645351 +0000 UTC m=+1549.248872882" Sep 19 06:07:01.624452 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:01.624384 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:02.248336 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:02.248301 2584 generic.go:356] "Generic (PLEG): container finished" podID="0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8" containerID="69b919ad0986c7ed56aa4b68da2ee0550a176ebc6532eba6e8c9d5bcf966727e" exitCode=0 Sep 19 06:07:02.248513 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:02.248382 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95" event={"ID":"0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8","Type":"ContainerDied","Data":"69b919ad0986c7ed56aa4b68da2ee0550a176ebc6532eba6e8c9d5bcf966727e"} Sep 19 06:07:02.486244 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:02.486195 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:03.263046 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:03.263002 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:03.392200 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:03.392173 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95" Sep 19 06:07:03.495012 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:03.494971 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8-maas-api-service-ca\") pod \"0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8\" (UID: \"0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8\") " Sep 19 06:07:03.495346 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:03.495323 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8" (UID: "0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:07:03.595787 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:03.595756 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:04.003052 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:04.002960 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:04.263583 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:04.263490 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95" Sep 19 06:07:04.268803 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:04.268762 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95" event={"ID":"0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8","Type":"ContainerDied","Data":"898d4b3a2b214a136bb16c21ca1d6818648a42f9c26d23ba2bdedb59d8b5e632"} Sep 19 06:07:04.268971 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:04.268822 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="898d4b3a2b214a136bb16c21ca1d6818648a42f9c26d23ba2bdedb59d8b5e632" Sep 19 06:07:04.646126 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:04.646091 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:05.698355 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:05.698324 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:05.935426 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:05.935365 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k"] Sep 19 06:07:05.935936 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:05.935921 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8" containerName="revoke-keys" Sep 19 06:07:05.935994 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:05.935937 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8" containerName="revoke-keys" Sep 19 06:07:05.936039 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:05.936007 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8" containerName="revoke-keys" Sep 19 06:07:05.939243 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:05.939227 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k" Sep 19 06:07:05.942192 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:05.942161 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:07:05.944709 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:05.944681 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k"] Sep 19 06:07:06.020169 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:06.020075 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34dd77ae-5b31-4f90-b089-0ab3baed941f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k\" (UID: \"34dd77ae-5b31-4f90-b089-0ab3baed941f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k" Sep 19 06:07:06.121414 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:06.121359 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34dd77ae-5b31-4f90-b089-0ab3baed941f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k\" (UID: \"34dd77ae-5b31-4f90-b089-0ab3baed941f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k" Sep 19 06:07:06.122219 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:06.122193 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34dd77ae-5b31-4f90-b089-0ab3baed941f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k\" (UID: \"34dd77ae-5b31-4f90-b089-0ab3baed941f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k" Sep 19 06:07:06.205833 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:06.205792 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:06.251960 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:06.251923 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k" Sep 19 06:07:06.534128 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:06.534099 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k"] Sep 19 06:07:06.535725 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:07:06.535698 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34dd77ae_5b31_4f90_b089_0ab3baed941f.slice/crio-9886bc77d6a1379b40be24cdfed9060139574c13dfed673b92e81bc1ef074c0d WatchSource:0}: Error finding container 9886bc77d6a1379b40be24cdfed9060139574c13dfed673b92e81bc1ef074c0d: Status 404 returned error can't find the container with id 9886bc77d6a1379b40be24cdfed9060139574c13dfed673b92e81bc1ef074c0d Sep 19 06:07:06.981033 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:06.980949 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:07.281083 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:07.281003 2584 generic.go:356] "Generic (PLEG): container finished" podID="34dd77ae-5b31-4f90-b089-0ab3baed941f" containerID="cc3106c58ef9f6d94c2a00efe19eee105652322a06197237e0928d5c69d0ce63" exitCode=0 Sep 19 06:07:07.281083 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:07.281042 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k" event={"ID":"34dd77ae-5b31-4f90-b089-0ab3baed941f","Type":"ContainerDied","Data":"cc3106c58ef9f6d94c2a00efe19eee105652322a06197237e0928d5c69d0ce63"} Sep 19 06:07:07.281277 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:07.281084 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k" event={"ID":"34dd77ae-5b31-4f90-b089-0ab3baed941f","Type":"ContainerStarted","Data":"9886bc77d6a1379b40be24cdfed9060139574c13dfed673b92e81bc1ef074c0d"} Sep 19 06:07:07.708146 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:07.708116 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:08.383870 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:08.383832 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:08.427572 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:08.427544 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k" Sep 19 06:07:08.547321 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:08.547232 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34dd77ae-5b31-4f90-b089-0ab3baed941f-maas-api-service-ca\") pod \"34dd77ae-5b31-4f90-b089-0ab3baed941f\" (UID: \"34dd77ae-5b31-4f90-b089-0ab3baed941f\") " Sep 19 06:07:08.547656 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:08.547633 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34dd77ae-5b31-4f90-b089-0ab3baed941f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "34dd77ae-5b31-4f90-b089-0ab3baed941f" (UID: "34dd77ae-5b31-4f90-b089-0ab3baed941f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:07:08.648681 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:08.648641 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34dd77ae-5b31-4f90-b089-0ab3baed941f-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:08.995163 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:08.995130 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:09.295099 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:09.295016 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k" Sep 19 06:07:09.295099 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:09.295050 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k" event={"ID":"34dd77ae-5b31-4f90-b089-0ab3baed941f","Type":"ContainerDied","Data":"9886bc77d6a1379b40be24cdfed9060139574c13dfed673b92e81bc1ef074c0d"} Sep 19 06:07:09.295099 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:09.295096 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9886bc77d6a1379b40be24cdfed9060139574c13dfed673b92e81bc1ef074c0d" Sep 19 06:07:09.673509 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:09.673473 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:10.296936 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:10.296899 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:10.760111 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:10.760077 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:11.023473 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.023373 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:11.049463 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.049432 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5"] Sep 19 06:07:11.049715 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.049687 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" podUID="a776fb4a-b357-4054-b367-ded97f53b90d" containerName="payload-processing" containerID="cri-o://9323d6833466799ccad972dcf1f05f2fe2b4feaff237d2f78e008692e8128fad" gracePeriod=30 Sep 19 06:07:11.120710 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.120681 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:11.202177 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.202113 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95"] Sep 19 06:07:11.203081 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.203049 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:11.205105 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.205076 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-2b3fb3-8qr95"] Sep 19 06:07:11.254995 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.254963 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:11.312174 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.312136 2584 generic.go:356] "Generic (PLEG): container finished" podID="a776fb4a-b357-4054-b367-ded97f53b90d" containerID="9323d6833466799ccad972dcf1f05f2fe2b4feaff237d2f78e008692e8128fad" exitCode=0 Sep 19 06:07:11.312324 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.312206 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" event={"ID":"a776fb4a-b357-4054-b367-ded97f53b90d","Type":"ContainerDied","Data":"9323d6833466799ccad972dcf1f05f2fe2b4feaff237d2f78e008692e8128fad"} Sep 19 06:07:11.312324 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.312276 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" event={"ID":"a776fb4a-b357-4054-b367-ded97f53b90d","Type":"ContainerDied","Data":"d10c5e7d87bc25a8eb8f2304293e77999f01b9a2fa789ca9b06db281aa207f28"} Sep 19 06:07:11.312324 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.312299 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d10c5e7d87bc25a8eb8f2304293e77999f01b9a2fa789ca9b06db281aa207f28" Sep 19 06:07:11.326657 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.326625 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" Sep 19 06:07:11.478479 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.478443 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cs5cq\" (UniqueName: \"kubernetes.io/projected/a776fb4a-b357-4054-b367-ded97f53b90d-kube-api-access-cs5cq\") pod \"a776fb4a-b357-4054-b367-ded97f53b90d\" (UID: \"a776fb4a-b357-4054-b367-ded97f53b90d\") " Sep 19 06:07:11.478647 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.478510 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a776fb4a-b357-4054-b367-ded97f53b90d-plugins-config-volume\") pod \"a776fb4a-b357-4054-b367-ded97f53b90d\" (UID: \"a776fb4a-b357-4054-b367-ded97f53b90d\") " Sep 19 06:07:11.478848 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.478820 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a776fb4a-b357-4054-b367-ded97f53b90d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a776fb4a-b357-4054-b367-ded97f53b90d" (UID: "a776fb4a-b357-4054-b367-ded97f53b90d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:07:11.480646 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.480629 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a776fb4a-b357-4054-b367-ded97f53b90d-kube-api-access-cs5cq" (OuterVolumeSpecName: "kube-api-access-cs5cq") pod "a776fb4a-b357-4054-b367-ded97f53b90d" (UID: "a776fb4a-b357-4054-b367-ded97f53b90d"). InnerVolumeSpecName "kube-api-access-cs5cq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:07:11.579512 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.579382 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a776fb4a-b357-4054-b367-ded97f53b90d-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:11.579512 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.579446 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cs5cq\" (UniqueName: \"kubernetes.io/projected/a776fb4a-b357-4054-b367-ded97f53b90d-kube-api-access-cs5cq\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:11.582066 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.582037 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:11.692343 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:11.692310 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:12.268654 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:12.268625 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8" path="/var/lib/kubelet/pods/0aa8ce47-0199-4231-8bbb-f2e6d81ed4c8/volumes" Sep 19 06:07:12.318656 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:12.318625 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5" Sep 19 06:07:12.339942 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:12.339901 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5"] Sep 19 06:07:12.344110 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:12.344067 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2b3fb3-6cdcd4f49-qfbn5"] Sep 19 06:07:12.360142 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:12.360112 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:13.002327 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:13.002298 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:13.654551 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:13.654497 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:14.271259 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:14.271222 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a776fb4a-b357-4054-b367-ded97f53b90d" path="/var/lib/kubelet/pods/a776fb4a-b357-4054-b367-ded97f53b90d/volumes" Sep 19 06:07:14.325160 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:14.325124 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:14.592299 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:14.592268 2584 scope.go:117] "RemoveContainer" containerID="76506b862742f0af142b6d41db4a09dd6c69749ed1239e3c7c95bf229a8026c5" Sep 19 06:07:14.601610 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:14.601588 2584 scope.go:117] "RemoveContainer" containerID="9323d6833466799ccad972dcf1f05f2fe2b4feaff237d2f78e008692e8128fad" Sep 19 06:07:16.290439 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:16.290385 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:16.982025 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:16.981971 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:17.508609 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.508573 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:17.604881 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.604810 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:17.635182 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.635136 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6"] Sep 19 06:07:17.637430 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.637408 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-3182b5-29829960-6rdm6"] Sep 19 06:07:17.658478 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.658443 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf"] Sep 19 06:07:17.658771 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.658734 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" podUID="24e5ff24-41b0-4fb4-a2c6-f46774d8f55b" containerName="payload-processing" containerID="cri-o://436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d" gracePeriod=30 Sep 19 06:07:17.670394 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.670364 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7"] Sep 19 06:07:17.670687 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.670646 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" podUID="a92660ef-fcfa-427d-98c1-f042db44e380" containerName="payload-pre-processing" containerID="cri-o://04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f" gracePeriod=30 Sep 19 06:07:17.673807 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.673779 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:17.703385 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.703354 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8102-a3a671"] Sep 19 06:07:17.703898 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.703879 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34dd77ae-5b31-4f90-b089-0ab3baed941f" containerName="revoke-keys" Sep 19 06:07:17.703984 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.703900 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="34dd77ae-5b31-4f90-b089-0ab3baed941f" containerName="revoke-keys" Sep 19 06:07:17.703984 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.703934 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a776fb4a-b357-4054-b367-ded97f53b90d" containerName="payload-processing" Sep 19 06:07:17.703984 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.703944 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a776fb4a-b357-4054-b367-ded97f53b90d" containerName="payload-processing" Sep 19 06:07:17.704145 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.704046 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="34dd77ae-5b31-4f90-b089-0ab3baed941f" containerName="revoke-keys" Sep 19 06:07:17.704145 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.704062 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a776fb4a-b357-4054-b367-ded97f53b90d" containerName="payload-processing" Sep 19 06:07:17.707362 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.707341 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8102-a3a671" Sep 19 06:07:17.710187 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.710155 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tpl82\"" Sep 19 06:07:17.713049 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.713026 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8102-a3a671"] Sep 19 06:07:17.776349 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.776225 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:17.777532 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.777346 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9"] Sep 19 06:07:17.777668 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.777637 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" containerID="cri-o://1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37" gracePeriod=30 Sep 19 06:07:17.839890 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.839855 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw5sp\" (UniqueName: \"kubernetes.io/projected/dfbddb13-a192-4ac7-80c0-d41823457344-kube-api-access-mw5sp\") pod \"test-curl-8102-a3a671\" (UID: \"dfbddb13-a192-4ac7-80c0-d41823457344\") " pod="opendatahub/test-curl-8102-a3a671" Sep 19 06:07:17.853999 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.853970 2584 patch_prober.go:28] interesting pod/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 19 06:07:17.854157 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.854074 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 19 06:07:17.871530 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.871499 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:17.877536 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.877509 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k"] Sep 19 06:07:17.883459 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.883435 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-3182b5-xtj5k"] Sep 19 06:07:17.952424 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.944117 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw5sp\" (UniqueName: \"kubernetes.io/projected/dfbddb13-a192-4ac7-80c0-d41823457344-kube-api-access-mw5sp\") pod \"test-curl-8102-a3a671\" (UID: \"dfbddb13-a192-4ac7-80c0-d41823457344\") " pod="opendatahub/test-curl-8102-a3a671" Sep 19 06:07:17.957853 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.957824 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw5sp\" (UniqueName: \"kubernetes.io/projected/dfbddb13-a192-4ac7-80c0-d41823457344-kube-api-access-mw5sp\") pod \"test-curl-8102-a3a671\" (UID: \"dfbddb13-a192-4ac7-80c0-d41823457344\") " pod="opendatahub/test-curl-8102-a3a671" Sep 19 06:07:17.970849 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.970826 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:17.984543 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.984521 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" Sep 19 06:07:17.986569 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:17.986554 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" Sep 19 06:07:18.117107 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.117071 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8102-a3a671" Sep 19 06:07:18.146360 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.146258 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rqlh\" (UniqueName: \"kubernetes.io/projected/a92660ef-fcfa-427d-98c1-f042db44e380-kube-api-access-2rqlh\") pod \"a92660ef-fcfa-427d-98c1-f042db44e380\" (UID: \"a92660ef-fcfa-427d-98c1-f042db44e380\") " Sep 19 06:07:18.146550 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.146448 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a92660ef-fcfa-427d-98c1-f042db44e380-plugins-config-volume\") pod \"a92660ef-fcfa-427d-98c1-f042db44e380\" (UID: \"a92660ef-fcfa-427d-98c1-f042db44e380\") " Sep 19 06:07:18.146550 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.146491 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f2vr\" (UniqueName: \"kubernetes.io/projected/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b-kube-api-access-4f2vr\") pod \"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b\" (UID: \"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b\") " Sep 19 06:07:18.146675 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.146531 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b-plugins-config-volume\") pod \"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b\" (UID: \"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b\") " Sep 19 06:07:18.147240 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.147206 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "24e5ff24-41b0-4fb4-a2c6-f46774d8f55b" (UID: "24e5ff24-41b0-4fb4-a2c6-f46774d8f55b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:07:18.147374 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.147256 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a92660ef-fcfa-427d-98c1-f042db44e380-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a92660ef-fcfa-427d-98c1-f042db44e380" (UID: "a92660ef-fcfa-427d-98c1-f042db44e380"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:07:18.148835 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.148808 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7"] Sep 19 06:07:18.148977 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.148956 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a92660ef-fcfa-427d-98c1-f042db44e380-kube-api-access-2rqlh" (OuterVolumeSpecName: "kube-api-access-2rqlh") pod "a92660ef-fcfa-427d-98c1-f042db44e380" (UID: "a92660ef-fcfa-427d-98c1-f042db44e380"). InnerVolumeSpecName "kube-api-access-2rqlh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:07:18.149025 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.148992 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b-kube-api-access-4f2vr" (OuterVolumeSpecName: "kube-api-access-4f2vr") pod "24e5ff24-41b0-4fb4-a2c6-f46774d8f55b" (UID: "24e5ff24-41b0-4fb4-a2c6-f46774d8f55b"). InnerVolumeSpecName "kube-api-access-4f2vr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:07:18.149310 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.149297 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24e5ff24-41b0-4fb4-a2c6-f46774d8f55b" containerName="payload-processing" Sep 19 06:07:18.149346 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.149312 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="24e5ff24-41b0-4fb4-a2c6-f46774d8f55b" containerName="payload-processing" Sep 19 06:07:18.149346 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.149323 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a92660ef-fcfa-427d-98c1-f042db44e380" containerName="payload-pre-processing" Sep 19 06:07:18.149346 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.149328 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a92660ef-fcfa-427d-98c1-f042db44e380" containerName="payload-pre-processing" Sep 19 06:07:18.149467 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.149436 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a92660ef-fcfa-427d-98c1-f042db44e380" containerName="payload-pre-processing" Sep 19 06:07:18.149467 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.149447 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="24e5ff24-41b0-4fb4-a2c6-f46774d8f55b" containerName="payload-processing" Sep 19 06:07:18.153675 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.153659 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7" Sep 19 06:07:18.156301 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.156275 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:07:18.159444 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.159365 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7"] Sep 19 06:07:18.239358 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.239330 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:18.247585 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.247554 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/372e16b7-02f8-45cd-9846-6f32c28343e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7\" (UID: \"372e16b7-02f8-45cd-9846-6f32c28343e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7" Sep 19 06:07:18.247737 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.247693 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a92660ef-fcfa-427d-98c1-f042db44e380-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:18.247737 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.247714 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f2vr\" (UniqueName: \"kubernetes.io/projected/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b-kube-api-access-4f2vr\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:18.247737 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.247724 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:18.247737 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.247732 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rqlh\" (UniqueName: \"kubernetes.io/projected/a92660ef-fcfa-427d-98c1-f042db44e380-kube-api-access-2rqlh\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:18.268240 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.268212 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34dd77ae-5b31-4f90-b089-0ab3baed941f" path="/var/lib/kubelet/pods/34dd77ae-5b31-4f90-b089-0ab3baed941f/volumes" Sep 19 06:07:18.268585 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.268570 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="856d5555-fb51-4f0b-b153-7d8d48a8b2a3" path="/var/lib/kubelet/pods/856d5555-fb51-4f0b-b153-7d8d48a8b2a3/volumes" Sep 19 06:07:18.338763 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.338732 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:18.348833 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.348806 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/372e16b7-02f8-45cd-9846-6f32c28343e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7\" (UID: \"372e16b7-02f8-45cd-9846-6f32c28343e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7" Sep 19 06:07:18.349395 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.349377 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/372e16b7-02f8-45cd-9846-6f32c28343e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7\" (UID: \"372e16b7-02f8-45cd-9846-6f32c28343e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7" Sep 19 06:07:18.357928 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.357902 2584 generic.go:356] "Generic (PLEG): container finished" podID="a92660ef-fcfa-427d-98c1-f042db44e380" containerID="04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f" exitCode=0 Sep 19 06:07:18.358060 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.357958 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" Sep 19 06:07:18.358060 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.357981 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" event={"ID":"a92660ef-fcfa-427d-98c1-f042db44e380","Type":"ContainerDied","Data":"04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f"} Sep 19 06:07:18.358166 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.358090 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7" event={"ID":"a92660ef-fcfa-427d-98c1-f042db44e380","Type":"ContainerDied","Data":"bdb3430eb18bd730ce8673c032db17b1f0ef4b975ae4ebe128cfa643afdf3b81"} Sep 19 06:07:18.358166 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.358117 2584 scope.go:117] "RemoveContainer" containerID="04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f" Sep 19 06:07:18.359530 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.359504 2584 generic.go:356] "Generic (PLEG): container finished" podID="24e5ff24-41b0-4fb4-a2c6-f46774d8f55b" containerID="436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d" exitCode=0 Sep 19 06:07:18.359659 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.359539 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" event={"ID":"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b","Type":"ContainerDied","Data":"436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d"} Sep 19 06:07:18.359659 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.359581 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" event={"ID":"24e5ff24-41b0-4fb4-a2c6-f46774d8f55b","Type":"ContainerDied","Data":"9146bbf80d5a6095dbbfaf545e8512dcac713a5338979589eec1ea326c969aa1"} Sep 19 06:07:18.359659 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.359596 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf" Sep 19 06:07:18.368067 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.368040 2584 scope.go:117] "RemoveContainer" containerID="04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f" Sep 19 06:07:18.368318 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:07:18.368293 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f\": container with ID starting with 04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f not found: ID does not exist" containerID="04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f" Sep 19 06:07:18.368433 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.368321 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f"} err="failed to get container status \"04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f\": rpc error: code = NotFound desc = could not find container \"04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f\": container with ID starting with 04a913009020436233ef35d034be7e993ae41106c157d22391825fa3613def2f not found: ID does not exist" Sep 19 06:07:18.368433 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.368337 2584 scope.go:117] "RemoveContainer" containerID="436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d" Sep 19 06:07:18.376833 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.376811 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7"] Sep 19 06:07:18.378076 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.378061 2584 scope.go:117] "RemoveContainer" containerID="436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d" Sep 19 06:07:18.378334 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:07:18.378316 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d\": container with ID starting with 436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d not found: ID does not exist" containerID="436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d" Sep 19 06:07:18.378410 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.378343 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d"} err="failed to get container status \"436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d\": rpc error: code = NotFound desc = could not find container \"436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d\": container with ID starting with 436946a8f2cedc857a2c9f97b384579c372e9c25a5a6256f9d1aefd6224e6b3d not found: ID does not exist" Sep 19 06:07:18.379758 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.379738 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-3182b5-8d78cc9cd-vdkm7"] Sep 19 06:07:18.387202 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.387182 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf"] Sep 19 06:07:18.390625 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.390604 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-3182b5-b54fdf565-8hzxf"] Sep 19 06:07:18.396363 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.396346 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8102-a3a671"] Sep 19 06:07:18.398499 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:07:18.398476 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfbddb13_a192_4ac7_80c0_d41823457344.slice/crio-8b16b1885623677b6409bba018322d89a3902c6116f8b1a80bdcab0ec6c30d9b WatchSource:0}: Error finding container 8b16b1885623677b6409bba018322d89a3902c6116f8b1a80bdcab0ec6c30d9b: Status 404 returned error can't find the container with id 8b16b1885623677b6409bba018322d89a3902c6116f8b1a80bdcab0ec6c30d9b Sep 19 06:07:18.470286 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.470263 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7" Sep 19 06:07:18.742041 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:18.742015 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7"] Sep 19 06:07:18.743547 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:07:18.743519 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372e16b7_02f8_45cd_9846_6f32c28343e2.slice/crio-0a7ecfc4dbfd90bde15c0a8c66085745a0690f6783172718be67bef3d89c0782 WatchSource:0}: Error finding container 0a7ecfc4dbfd90bde15c0a8c66085745a0690f6783172718be67bef3d89c0782: Status 404 returned error can't find the container with id 0a7ecfc4dbfd90bde15c0a8c66085745a0690f6783172718be67bef3d89c0782 Sep 19 06:07:19.022523 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:19.022498 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:19.367015 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:19.366981 2584 generic.go:356] "Generic (PLEG): container finished" podID="372e16b7-02f8-45cd-9846-6f32c28343e2" containerID="85db361ea6d95de1218a5b763a91cb08228c685e7f274ea2b04d75013bb7efd1" exitCode=0 Sep 19 06:07:19.367176 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:19.367055 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7" event={"ID":"372e16b7-02f8-45cd-9846-6f32c28343e2","Type":"ContainerDied","Data":"85db361ea6d95de1218a5b763a91cb08228c685e7f274ea2b04d75013bb7efd1"} Sep 19 06:07:19.367176 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:19.367117 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7" event={"ID":"372e16b7-02f8-45cd-9846-6f32c28343e2","Type":"ContainerStarted","Data":"0a7ecfc4dbfd90bde15c0a8c66085745a0690f6783172718be67bef3d89c0782"} Sep 19 06:07:19.368656 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:19.368630 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8102-a3a671" event={"ID":"dfbddb13-a192-4ac7-80c0-d41823457344","Type":"ContainerStarted","Data":"2a6e5c86e20d71130e8c4b53711594c9ce71e14bc1edb1c1268085351443981f"} Sep 19 06:07:19.368809 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:19.368672 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8102-a3a671" event={"ID":"dfbddb13-a192-4ac7-80c0-d41823457344","Type":"ContainerStarted","Data":"8b16b1885623677b6409bba018322d89a3902c6116f8b1a80bdcab0ec6c30d9b"} Sep 19 06:07:19.394782 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:19.394732 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8102-a3a671" podStartSLOduration=2.394718593 podStartE2EDuration="2.394718593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:07:19.392810768 +0000 UTC m=+1567.386038303" watchObservedRunningTime="2026-09-19 06:07:19.394718593 +0000 UTC m=+1567.387946124" Sep 19 06:07:19.784011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:19.783905 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:20.009102 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:20.009065 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8102-a3a671"] Sep 19 06:07:20.014198 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:20.014168 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8102-a3a671"] Sep 19 06:07:20.267253 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:20.267214 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24e5ff24-41b0-4fb4-a2c6-f46774d8f55b" path="/var/lib/kubelet/pods/24e5ff24-41b0-4fb4-a2c6-f46774d8f55b/volumes" Sep 19 06:07:20.267680 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:20.267658 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a92660ef-fcfa-427d-98c1-f042db44e380" path="/var/lib/kubelet/pods/a92660ef-fcfa-427d-98c1-f042db44e380/volumes" Sep 19 06:07:20.376797 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:07:20.376767 2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8102-a3a671" container="test-curl-8102-a3a671" Sep 19 06:07:20.507026 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:20.506979 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:20.517837 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:20.517776 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7" Sep 19 06:07:20.671710 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:20.671675 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/372e16b7-02f8-45cd-9846-6f32c28343e2-maas-api-service-ca\") pod \"372e16b7-02f8-45cd-9846-6f32c28343e2\" (UID: \"372e16b7-02f8-45cd-9846-6f32c28343e2\") " Sep 19 06:07:20.672112 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:20.672086 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/372e16b7-02f8-45cd-9846-6f32c28343e2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "372e16b7-02f8-45cd-9846-6f32c28343e2" (UID: "372e16b7-02f8-45cd-9846-6f32c28343e2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:07:20.773462 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:20.773359 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/372e16b7-02f8-45cd-9846-6f32c28343e2-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:21.194932 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:21.194903 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:21.384372 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:21.384345 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7" Sep 19 06:07:21.384551 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:21.384390 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7" event={"ID":"372e16b7-02f8-45cd-9846-6f32c28343e2","Type":"ContainerDied","Data":"0a7ecfc4dbfd90bde15c0a8c66085745a0690f6783172718be67bef3d89c0782"} Sep 19 06:07:21.384551 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:21.384454 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a7ecfc4dbfd90bde15c0a8c66085745a0690f6783172718be67bef3d89c0782" Sep 19 06:07:21.384641 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:21.384622 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8102-a3a671" podUID="dfbddb13-a192-4ac7-80c0-d41823457344" containerName="test-curl-8102-a3a671" containerID="cri-o://2a6e5c86e20d71130e8c4b53711594c9ce71e14bc1edb1c1268085351443981f" gracePeriod=2 Sep 19 06:07:22.256826 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:22.256794 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:23.034969 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.034944 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 06:07:23.091871 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.091831 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/195b28ee-0890-40b9-854f-c8a507da24a1-istio-podinfo\") pod \"195b28ee-0890-40b9-854f-c8a507da24a1\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " Sep 19 06:07:23.092011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.091921 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-credential-socket\") pod \"195b28ee-0890-40b9-854f-c8a507da24a1\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " Sep 19 06:07:23.092011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.091961 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wsqf\" (UniqueName: \"kubernetes.io/projected/195b28ee-0890-40b9-854f-c8a507da24a1-kube-api-access-9wsqf\") pod \"195b28ee-0890-40b9-854f-c8a507da24a1\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " Sep 19 06:07:23.092011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.092000 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/195b28ee-0890-40b9-854f-c8a507da24a1-istio-token\") pod \"195b28ee-0890-40b9-854f-c8a507da24a1\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " Sep 19 06:07:23.092159 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.092041 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-istio-envoy\") pod \"195b28ee-0890-40b9-854f-c8a507da24a1\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " Sep 19 06:07:23.092159 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.092058 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/195b28ee-0890-40b9-854f-c8a507da24a1-istiod-ca-cert\") pod \"195b28ee-0890-40b9-854f-c8a507da24a1\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " Sep 19 06:07:23.092263 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.092162 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-istio-data\") pod \"195b28ee-0890-40b9-854f-c8a507da24a1\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " Sep 19 06:07:23.092263 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.092201 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-workload-socket\") pod \"195b28ee-0890-40b9-854f-c8a507da24a1\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " Sep 19 06:07:23.092263 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.092229 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-workload-certs\") pod \"195b28ee-0890-40b9-854f-c8a507da24a1\" (UID: \"195b28ee-0890-40b9-854f-c8a507da24a1\") " Sep 19 06:07:23.092472 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.092265 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "195b28ee-0890-40b9-854f-c8a507da24a1" (UID: "195b28ee-0890-40b9-854f-c8a507da24a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:07:23.092654 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.092567 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/195b28ee-0890-40b9-854f-c8a507da24a1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "195b28ee-0890-40b9-854f-c8a507da24a1" (UID: "195b28ee-0890-40b9-854f-c8a507da24a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:07:23.092788 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.092693 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-istio-data" (OuterVolumeSpecName: "istio-data") pod "195b28ee-0890-40b9-854f-c8a507da24a1" (UID: "195b28ee-0890-40b9-854f-c8a507da24a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:07:23.092788 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.092775 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "195b28ee-0890-40b9-854f-c8a507da24a1" (UID: "195b28ee-0890-40b9-854f-c8a507da24a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:07:23.092914 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.092789 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "195b28ee-0890-40b9-854f-c8a507da24a1" (UID: "195b28ee-0890-40b9-854f-c8a507da24a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:07:23.093160 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.093131 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-credential-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:23.093265 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.093164 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/195b28ee-0890-40b9-854f-c8a507da24a1-istiod-ca-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:23.093265 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.093180 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-istio-data\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:23.093265 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.093194 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-workload-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:23.093265 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.093206 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-workload-certs\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:23.094333 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.094306 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/195b28ee-0890-40b9-854f-c8a507da24a1-istio-token" (OuterVolumeSpecName: "istio-token") pod "195b28ee-0890-40b9-854f-c8a507da24a1" (UID: "195b28ee-0890-40b9-854f-c8a507da24a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:07:23.094450 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.094423 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/195b28ee-0890-40b9-854f-c8a507da24a1-kube-api-access-9wsqf" (OuterVolumeSpecName: "kube-api-access-9wsqf") pod "195b28ee-0890-40b9-854f-c8a507da24a1" (UID: "195b28ee-0890-40b9-854f-c8a507da24a1"). InnerVolumeSpecName "kube-api-access-9wsqf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:07:23.094593 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.094570 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "195b28ee-0890-40b9-854f-c8a507da24a1" (UID: "195b28ee-0890-40b9-854f-c8a507da24a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:07:23.094593 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.094575 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/195b28ee-0890-40b9-854f-c8a507da24a1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "195b28ee-0890-40b9-854f-c8a507da24a1" (UID: "195b28ee-0890-40b9-854f-c8a507da24a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 06:07:23.194506 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.194461 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/195b28ee-0890-40b9-854f-c8a507da24a1-istio-podinfo\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:23.194506 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.194493 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wsqf\" (UniqueName: \"kubernetes.io/projected/195b28ee-0890-40b9-854f-c8a507da24a1-kube-api-access-9wsqf\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:23.194506 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.194502 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/195b28ee-0890-40b9-854f-c8a507da24a1-istio-token\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:23.194506 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.194513 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/195b28ee-0890-40b9-854f-c8a507da24a1-istio-envoy\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:23.399202 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.399175 2584 generic.go:356] "Generic (PLEG): container finished" podID="195b28ee-0890-40b9-854f-c8a507da24a1" containerID="1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37" exitCode=0 Sep 19 06:07:23.399714 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.399222 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" Sep 19 06:07:23.399714 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.399267 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" event={"ID":"195b28ee-0890-40b9-854f-c8a507da24a1","Type":"ContainerDied","Data":"1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37"} Sep 19 06:07:23.399714 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.399327 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9" event={"ID":"195b28ee-0890-40b9-854f-c8a507da24a1","Type":"ContainerDied","Data":"f8200b5d1aeb8cee5059aeda47e61d0e49ffb7ddb4f51282849517bdb9fd1ee6"} Sep 19 06:07:23.399714 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.399352 2584 scope.go:117] "RemoveContainer" containerID="1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37" Sep 19 06:07:23.444797 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.444766 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:23.453534 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.453515 2584 scope.go:117] "RemoveContainer" containerID="6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b" Sep 19 06:07:23.462963 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.462938 2584 scope.go:117] "RemoveContainer" containerID="1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37" Sep 19 06:07:23.463634 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:07:23.463324 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37\": container with ID starting with 1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37 not found: ID does not exist" containerID="1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37" Sep 19 06:07:23.463634 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.463362 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37"} err="failed to get container status \"1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37\": rpc error: code = NotFound desc = could not find container \"1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37\": container with ID starting with 1881d418c6cac1eeea453bb67686d90297fcb506e8d27b40d4ed72f0102e0d37 not found: ID does not exist" Sep 19 06:07:23.463634 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.463387 2584 scope.go:117] "RemoveContainer" containerID="6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b" Sep 19 06:07:23.463835 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:07:23.463790 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b\": container with ID starting with 6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b not found: ID does not exist" containerID="6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b" Sep 19 06:07:23.463878 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.463828 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b"} err="failed to get container status \"6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b\": rpc error: code = NotFound desc = could not find container \"6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b\": container with ID starting with 6b7161c224e5ee77a96ac1167f3d07208de4012489b37c08ee83105031efca8b not found: ID does not exist" Sep 19 06:07:23.464268 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.464248 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9"] Sep 19 06:07:23.472213 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.472187 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2b3fb3-openshift-default-779c455c58-lb4v9"] Sep 19 06:07:23.530806 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.530781 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8102-a3a671" Sep 19 06:07:23.598237 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.598202 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw5sp\" (UniqueName: \"kubernetes.io/projected/dfbddb13-a192-4ac7-80c0-d41823457344-kube-api-access-mw5sp\") pod \"dfbddb13-a192-4ac7-80c0-d41823457344\" (UID: \"dfbddb13-a192-4ac7-80c0-d41823457344\") " Sep 19 06:07:23.600289 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.600266 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfbddb13-a192-4ac7-80c0-d41823457344-kube-api-access-mw5sp" (OuterVolumeSpecName: "kube-api-access-mw5sp") pod "dfbddb13-a192-4ac7-80c0-d41823457344" (UID: "dfbddb13-a192-4ac7-80c0-d41823457344"). InnerVolumeSpecName "kube-api-access-mw5sp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:07:23.700006 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.699921 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw5sp\" (UniqueName: \"kubernetes.io/projected/dfbddb13-a192-4ac7-80c0-d41823457344-kube-api-access-mw5sp\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:23.926049 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.926010 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:23.979874 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:23.979783 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:24.196855 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:24.196819 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:24.270060 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:24.269980 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" path="/var/lib/kubelet/pods/195b28ee-0890-40b9-854f-c8a507da24a1/volumes" Sep 19 06:07:24.270356 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:24.270343 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfbddb13-a192-4ac7-80c0-d41823457344" path="/var/lib/kubelet/pods/dfbddb13-a192-4ac7-80c0-d41823457344/volumes" Sep 19 06:07:24.405907 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:24.405875 2584 generic.go:356] "Generic (PLEG): container finished" podID="dfbddb13-a192-4ac7-80c0-d41823457344" containerID="2a6e5c86e20d71130e8c4b53711594c9ce71e14bc1edb1c1268085351443981f" exitCode=137 Sep 19 06:07:24.406340 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:24.405953 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8102-a3a671" Sep 19 06:07:24.406340 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:24.405981 2584 scope.go:117] "RemoveContainer" containerID="2a6e5c86e20d71130e8c4b53711594c9ce71e14bc1edb1c1268085351443981f" Sep 19 06:07:24.414885 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:24.414863 2584 scope.go:117] "RemoveContainer" containerID="2a6e5c86e20d71130e8c4b53711594c9ce71e14bc1edb1c1268085351443981f" Sep 19 06:07:24.415163 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:07:24.415142 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a6e5c86e20d71130e8c4b53711594c9ce71e14bc1edb1c1268085351443981f\": container with ID starting with 2a6e5c86e20d71130e8c4b53711594c9ce71e14bc1edb1c1268085351443981f not found: ID does not exist" containerID="2a6e5c86e20d71130e8c4b53711594c9ce71e14bc1edb1c1268085351443981f" Sep 19 06:07:24.415239 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:24.415171 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a6e5c86e20d71130e8c4b53711594c9ce71e14bc1edb1c1268085351443981f"} err="failed to get container status \"2a6e5c86e20d71130e8c4b53711594c9ce71e14bc1edb1c1268085351443981f\": rpc error: code = NotFound desc = could not find container \"2a6e5c86e20d71130e8c4b53711594c9ce71e14bc1edb1c1268085351443981f\": container with ID starting with 2a6e5c86e20d71130e8c4b53711594c9ce71e14bc1edb1c1268085351443981f not found: ID does not exist" Sep 19 06:07:28.405804 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:28.405730 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:28.955421 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:28.955375 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:28.956101 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:28.956074 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs"] Sep 19 06:07:28.958123 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:28.958104 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-836c16-29829960-d66rs"] Sep 19 06:07:28.988059 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:28.988019 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj"] Sep 19 06:07:28.988320 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:28.988272 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" podUID="7d0d1284-2493-4e0e-be3e-771d73140c45" containerName="payload-pre-processing" containerID="cri-o://b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0" gracePeriod=30 Sep 19 06:07:29.018745 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.018719 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:29.119665 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.119631 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:29.138369 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.138329 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7"] Sep 19 06:07:29.140322 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.140294 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-836c16-zwjt7"] Sep 19 06:07:29.173178 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.173154 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:29.238945 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.238915 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" Sep 19 06:07:29.355827 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.355790 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xz8xw\" (UniqueName: \"kubernetes.io/projected/7d0d1284-2493-4e0e-be3e-771d73140c45-kube-api-access-xz8xw\") pod \"7d0d1284-2493-4e0e-be3e-771d73140c45\" (UID: \"7d0d1284-2493-4e0e-be3e-771d73140c45\") " Sep 19 06:07:29.356024 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.355852 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0d1284-2493-4e0e-be3e-771d73140c45-plugins-config-volume\") pod \"7d0d1284-2493-4e0e-be3e-771d73140c45\" (UID: \"7d0d1284-2493-4e0e-be3e-771d73140c45\") " Sep 19 06:07:29.356245 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.356219 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d0d1284-2493-4e0e-be3e-771d73140c45-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d0d1284-2493-4e0e-be3e-771d73140c45" (UID: "7d0d1284-2493-4e0e-be3e-771d73140c45"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:07:29.357908 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.357887 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d0d1284-2493-4e0e-be3e-771d73140c45-kube-api-access-xz8xw" (OuterVolumeSpecName: "kube-api-access-xz8xw") pod "7d0d1284-2493-4e0e-be3e-771d73140c45" (UID: "7d0d1284-2493-4e0e-be3e-771d73140c45"). InnerVolumeSpecName "kube-api-access-xz8xw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:07:29.438691 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.438654 2584 generic.go:356] "Generic (PLEG): container finished" podID="7d0d1284-2493-4e0e-be3e-771d73140c45" containerID="b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0" exitCode=0 Sep 19 06:07:29.439093 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.438697 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" Sep 19 06:07:29.439093 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.438741 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" event={"ID":"7d0d1284-2493-4e0e-be3e-771d73140c45","Type":"ContainerDied","Data":"b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0"} Sep 19 06:07:29.439093 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.438788 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj" event={"ID":"7d0d1284-2493-4e0e-be3e-771d73140c45","Type":"ContainerDied","Data":"551b1257661e9a71a3689d72dd4e44e9ef8f924cbe8c9ac5f6eece8f5241b8ed"} Sep 19 06:07:29.439093 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.438811 2584 scope.go:117] "RemoveContainer" containerID="b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0" Sep 19 06:07:29.448127 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.448112 2584 scope.go:117] "RemoveContainer" containerID="b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0" Sep 19 06:07:29.448391 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:07:29.448375 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0\": container with ID starting with b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0 not found: ID does not exist" containerID="b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0" Sep 19 06:07:29.448529 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.448412 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0"} err="failed to get container status \"b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0\": rpc error: code = NotFound desc = could not find container \"b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0\": container with ID starting with b7ac910c26ab40450daaea15246d9ae21f70baa131f7454b6b5ff2f00a6cf6d0 not found: ID does not exist" Sep 19 06:07:29.456846 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.456817 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xz8xw\" (UniqueName: \"kubernetes.io/projected/7d0d1284-2493-4e0e-be3e-771d73140c45-kube-api-access-xz8xw\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:29.456846 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.456846 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0d1284-2493-4e0e-be3e-771d73140c45-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:07:29.458964 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.458937 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj"] Sep 19 06:07:29.464325 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.464292 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-836c16-67555b54-566rj"] Sep 19 06:07:29.677598 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.677562 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:29.778591 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:29.778554 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:30.220942 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:30.220911 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:30.267669 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:30.267633 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="372e16b7-02f8-45cd-9846-6f32c28343e2" path="/var/lib/kubelet/pods/372e16b7-02f8-45cd-9846-6f32c28343e2/volumes" Sep 19 06:07:30.268080 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:30.268060 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d0d1284-2493-4e0e-be3e-771d73140c45" path="/var/lib/kubelet/pods/7d0d1284-2493-4e0e-be3e-771d73140c45/volumes" Sep 19 06:07:30.268391 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:30.268373 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b304f2cf-92c2-4d7b-ae5c-2847d6fedba1" path="/var/lib/kubelet/pods/b304f2cf-92c2-4d7b-ae5c-2847d6fedba1/volumes" Sep 19 06:07:30.980317 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:30.980266 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:35.034548 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:35.034511 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:35.135230 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:35.135195 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:35.298555 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:35.298470 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:45.247478 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:45.247438 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:46.723732 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:46.723690 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:47.437786 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:47.437747 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:47.489139 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:47.489103 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:07:48.378631 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:07:48.378591 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:08:14.649814 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:08:14.649721 2584 scope.go:117] "RemoveContainer" containerID="8abbaaae1d6cf72b53b40625f3255fc0a22307e4e6ce01c4fe95e9df406b994b" Sep 19 06:08:14.658698 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:08:14.658678 2584 scope.go:117] "RemoveContainer" containerID="461b0a3f2f1b7b2205469c758aa27fe5860f795b1b95206bfa415230cbd00744" Sep 19 06:08:14.666887 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:08:14.666870 2584 scope.go:117] "RemoveContainer" containerID="078311a50e32d955f1edfe597914deee209fb48d2a84a59f93f63dee407ae71f" Sep 19 06:08:19.645186 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:08:19.645145 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:08:24.261442 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:08:24.261386 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:08:25.276945 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:08:25.276898 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:08:38.460420 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:08:38.460361 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:08:54.596928 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:08:54.596883 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:09:14.720957 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:14.720920 2584 scope.go:117] "RemoveContainer" containerID="0f33b817e9ec2245b3bcd32e6178788f3718d40af9af7310fdc1aff3d22aab57" Sep 19 06:09:16.673074 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673038 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8102-adcbd1"] Sep 19 06:09:16.673513 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673488 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfbddb13-a192-4ac7-80c0-d41823457344" containerName="test-curl-8102-a3a671" Sep 19 06:09:16.673513 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673501 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfbddb13-a192-4ac7-80c0-d41823457344" containerName="test-curl-8102-a3a671" Sep 19 06:09:16.673599 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673516 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="372e16b7-02f8-45cd-9846-6f32c28343e2" containerName="revoke-keys" Sep 19 06:09:16.673599 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673522 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="372e16b7-02f8-45cd-9846-6f32c28343e2" containerName="revoke-keys" Sep 19 06:09:16.673599 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673530 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" Sep 19 06:09:16.673599 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673536 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" Sep 19 06:09:16.673599 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673542 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d0d1284-2493-4e0e-be3e-771d73140c45" containerName="payload-pre-processing" Sep 19 06:09:16.673599 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673548 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d0d1284-2493-4e0e-be3e-771d73140c45" containerName="payload-pre-processing" Sep 19 06:09:16.673599 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673555 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" Sep 19 06:09:16.673599 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673561 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" Sep 19 06:09:16.673836 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673632 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d0d1284-2493-4e0e-be3e-771d73140c45" containerName="payload-pre-processing" Sep 19 06:09:16.673836 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673641 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="372e16b7-02f8-45cd-9846-6f32c28343e2" containerName="revoke-keys" Sep 19 06:09:16.673836 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673648 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfbddb13-a192-4ac7-80c0-d41823457344" containerName="test-curl-8102-a3a671" Sep 19 06:09:16.673836 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673656 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" Sep 19 06:09:16.673836 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.673661 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="195b28ee-0890-40b9-854f-c8a507da24a1" containerName="istio-proxy" Sep 19 06:09:16.678141 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.678119 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8102-adcbd1" Sep 19 06:09:16.680775 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.680742 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tpl82\"" Sep 19 06:09:16.682537 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.682511 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8102-adcbd1"] Sep 19 06:09:16.713783 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.713741 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r826\" (UniqueName: \"kubernetes.io/projected/50c0a89d-a1f8-41b4-ac80-543b46fece8b-kube-api-access-5r826\") pod \"test-curl-8102-adcbd1\" (UID: \"50c0a89d-a1f8-41b4-ac80-543b46fece8b\") " pod="opendatahub/test-curl-8102-adcbd1" Sep 19 06:09:16.814712 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.814667 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r826\" (UniqueName: \"kubernetes.io/projected/50c0a89d-a1f8-41b4-ac80-543b46fece8b-kube-api-access-5r826\") pod \"test-curl-8102-adcbd1\" (UID: \"50c0a89d-a1f8-41b4-ac80-543b46fece8b\") " pod="opendatahub/test-curl-8102-adcbd1" Sep 19 06:09:16.822115 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.822088 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r826\" (UniqueName: \"kubernetes.io/projected/50c0a89d-a1f8-41b4-ac80-543b46fece8b-kube-api-access-5r826\") pod \"test-curl-8102-adcbd1\" (UID: \"50c0a89d-a1f8-41b4-ac80-543b46fece8b\") " pod="opendatahub/test-curl-8102-adcbd1" Sep 19 06:09:16.989813 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:16.989729 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8102-adcbd1" Sep 19 06:09:17.260797 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:17.260767 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8102-adcbd1"] Sep 19 06:09:17.263034 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:09:17.262993 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50c0a89d_a1f8_41b4_ac80_543b46fece8b.slice/crio-6ae7b38012fc26472113ec5893b029a2b0efba9e324109be65d4a1001ee108ba WatchSource:0}: Error finding container 6ae7b38012fc26472113ec5893b029a2b0efba9e324109be65d4a1001ee108ba: Status 404 returned error can't find the container with id 6ae7b38012fc26472113ec5893b029a2b0efba9e324109be65d4a1001ee108ba Sep 19 06:09:18.041032 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.040994 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8102-adcbd1" event={"ID":"50c0a89d-a1f8-41b4-ac80-543b46fece8b","Type":"ContainerStarted","Data":"b7cbe11f8de30cb3c1fde9632d161e1573e0f4a9d1e8bda920ab217302eb3e8d"} Sep 19 06:09:18.041032 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.041037 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8102-adcbd1" event={"ID":"50c0a89d-a1f8-41b4-ac80-543b46fece8b","Type":"ContainerStarted","Data":"6ae7b38012fc26472113ec5893b029a2b0efba9e324109be65d4a1001ee108ba"} Sep 19 06:09:18.056381 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.056321 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8102-adcbd1" podStartSLOduration=2.056304477 podStartE2EDuration="2.056304477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:09:18.054068005 +0000 UTC m=+1686.047295537" watchObservedRunningTime="2026-09-19 06:09:18.056304477 +0000 UTC m=+1686.049532008" Sep 19 06:09:18.670249 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.670207 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8102-adcbd1"] Sep 19 06:09:18.673381 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.673354 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8102-adcbd1"] Sep 19 06:09:18.789885 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.789801 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8102-13da09"] Sep 19 06:09:18.790263 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.790250 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50c0a89d-a1f8-41b4-ac80-543b46fece8b" containerName="test-curl-8102-adcbd1" Sep 19 06:09:18.790307 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.790265 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="50c0a89d-a1f8-41b4-ac80-543b46fece8b" containerName="test-curl-8102-adcbd1" Sep 19 06:09:18.790359 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.790349 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="50c0a89d-a1f8-41b4-ac80-543b46fece8b" containerName="test-curl-8102-adcbd1" Sep 19 06:09:18.793315 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.793298 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8102-13da09" Sep 19 06:09:18.798896 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.798866 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8102-13da09"] Sep 19 06:09:18.833825 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.833781 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwjss\" (UniqueName: \"kubernetes.io/projected/cd5c484e-1e34-4cf7-a039-36f964438006-kube-api-access-qwjss\") pod \"test-curl-8102-13da09\" (UID: \"cd5c484e-1e34-4cf7-a039-36f964438006\") " pod="opendatahub/test-curl-8102-13da09" Sep 19 06:09:18.935300 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.935266 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwjss\" (UniqueName: \"kubernetes.io/projected/cd5c484e-1e34-4cf7-a039-36f964438006-kube-api-access-qwjss\") pod \"test-curl-8102-13da09\" (UID: \"cd5c484e-1e34-4cf7-a039-36f964438006\") " pod="opendatahub/test-curl-8102-13da09" Sep 19 06:09:18.942995 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:18.942969 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwjss\" (UniqueName: \"kubernetes.io/projected/cd5c484e-1e34-4cf7-a039-36f964438006-kube-api-access-qwjss\") pod \"test-curl-8102-13da09\" (UID: \"cd5c484e-1e34-4cf7-a039-36f964438006\") " pod="opendatahub/test-curl-8102-13da09" Sep 19 06:09:19.047529 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:09:19.047447 2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8102-adcbd1" container="test-curl-8102-adcbd1" Sep 19 06:09:19.104150 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:19.104113 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8102-13da09" Sep 19 06:09:19.373257 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:19.373226 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8102-13da09"] Sep 19 06:09:19.374674 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:09:19.374646 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd5c484e_1e34_4cf7_a039_36f964438006.slice/crio-50e3b11c8494ace891179fc84d09cd0297e96c35e3fbf663e25047a3f59cb003 WatchSource:0}: Error finding container 50e3b11c8494ace891179fc84d09cd0297e96c35e3fbf663e25047a3f59cb003: Status 404 returned error can't find the container with id 50e3b11c8494ace891179fc84d09cd0297e96c35e3fbf663e25047a3f59cb003 Sep 19 06:09:20.054691 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:20.054650 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8102-13da09" event={"ID":"cd5c484e-1e34-4cf7-a039-36f964438006","Type":"ContainerStarted","Data":"9214f0844be4f19233a703544d3478949ea6e2c15444f2aa8defe3f672095ac4"} Sep 19 06:09:20.054691 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:20.054701 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8102-13da09" event={"ID":"cd5c484e-1e34-4cf7-a039-36f964438006","Type":"ContainerStarted","Data":"50e3b11c8494ace891179fc84d09cd0297e96c35e3fbf663e25047a3f59cb003"} Sep 19 06:09:20.055267 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:20.054693 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8102-adcbd1" podUID="50c0a89d-a1f8-41b4-ac80-543b46fece8b" containerName="test-curl-8102-adcbd1" containerID="cri-o://b7cbe11f8de30cb3c1fde9632d161e1573e0f4a9d1e8bda920ab217302eb3e8d" gracePeriod=2 Sep 19 06:09:20.057013 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:09:20.056985 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8102-adcbd1\" is forbidden: User \"system:node:ip-10-0-142-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-57.ec2.internal' and this object" podUID="50c0a89d-a1f8-41b4-ac80-543b46fece8b" pod="opendatahub/test-curl-8102-adcbd1" Sep 19 06:09:20.072274 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:20.072218 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8102-13da09" podStartSLOduration=2.072203281 podStartE2EDuration="2.072203281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:09:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:09:20.071163902 +0000 UTC m=+1688.064391434" watchObservedRunningTime="2026-09-19 06:09:20.072203281 +0000 UTC m=+1688.065430860" Sep 19 06:09:20.073387 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:09:20.073357 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8102-adcbd1\" is forbidden: User \"system:node:ip-10-0-142-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-57.ec2.internal' and this object" podUID="50c0a89d-a1f8-41b4-ac80-543b46fece8b" pod="opendatahub/test-curl-8102-adcbd1" Sep 19 06:09:20.675708 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:20.675660 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8102-13da09"] Sep 19 06:09:20.678245 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:20.678202 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8102-13da09"] Sep 19 06:09:21.061086 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:09:21.061057 2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8102-13da09" container="test-curl-8102-13da09" Sep 19 06:09:22.066563 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:22.066531 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8102-13da09" podUID="cd5c484e-1e34-4cf7-a039-36f964438006" containerName="test-curl-8102-13da09" containerID="cri-o://9214f0844be4f19233a703544d3478949ea6e2c15444f2aa8defe3f672095ac4" gracePeriod=2 Sep 19 06:09:22.068541 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:09:22.068494 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8102-13da09\" is forbidden: User \"system:node:ip-10-0-142-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-57.ec2.internal' and this object" podUID="cd5c484e-1e34-4cf7-a039-36f964438006" pod="opendatahub/test-curl-8102-13da09" Sep 19 06:09:22.226863 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:22.226833 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8102-adcbd1" Sep 19 06:09:22.228895 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:09:22.228867 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8102-13da09\" is forbidden: User \"system:node:ip-10-0-142-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-57.ec2.internal' and this object" podUID="cd5c484e-1e34-4cf7-a039-36f964438006" pod="opendatahub/test-curl-8102-13da09" Sep 19 06:09:22.269181 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:22.269153 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5r826\" (UniqueName: \"kubernetes.io/projected/50c0a89d-a1f8-41b4-ac80-543b46fece8b-kube-api-access-5r826\") pod \"50c0a89d-a1f8-41b4-ac80-543b46fece8b\" (UID: \"50c0a89d-a1f8-41b4-ac80-543b46fece8b\") " Sep 19 06:09:22.271171 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:22.271143 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50c0a89d-a1f8-41b4-ac80-543b46fece8b-kube-api-access-5r826" (OuterVolumeSpecName: "kube-api-access-5r826") pod "50c0a89d-a1f8-41b4-ac80-543b46fece8b" (UID: "50c0a89d-a1f8-41b4-ac80-543b46fece8b"). InnerVolumeSpecName "kube-api-access-5r826". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:09:22.370992 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:22.370955 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r826\" (UniqueName: \"kubernetes.io/projected/50c0a89d-a1f8-41b4-ac80-543b46fece8b-kube-api-access-5r826\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:09:22.740819 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:22.740738 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:09:23.073165 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:23.073076 2584 generic.go:356] "Generic (PLEG): container finished" podID="50c0a89d-a1f8-41b4-ac80-543b46fece8b" containerID="b7cbe11f8de30cb3c1fde9632d161e1573e0f4a9d1e8bda920ab217302eb3e8d" exitCode=137 Sep 19 06:09:23.073165 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:23.073122 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8102-adcbd1" Sep 19 06:09:23.073620 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:23.073184 2584 scope.go:117] "RemoveContainer" containerID="b7cbe11f8de30cb3c1fde9632d161e1573e0f4a9d1e8bda920ab217302eb3e8d" Sep 19 06:09:23.083016 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:23.082995 2584 scope.go:117] "RemoveContainer" containerID="b7cbe11f8de30cb3c1fde9632d161e1573e0f4a9d1e8bda920ab217302eb3e8d" Sep 19 06:09:23.083323 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:09:23.083303 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7cbe11f8de30cb3c1fde9632d161e1573e0f4a9d1e8bda920ab217302eb3e8d\": container with ID starting with b7cbe11f8de30cb3c1fde9632d161e1573e0f4a9d1e8bda920ab217302eb3e8d not found: ID does not exist" containerID="b7cbe11f8de30cb3c1fde9632d161e1573e0f4a9d1e8bda920ab217302eb3e8d" Sep 19 06:09:23.083386 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:23.083333 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7cbe11f8de30cb3c1fde9632d161e1573e0f4a9d1e8bda920ab217302eb3e8d"} err="failed to get container status \"b7cbe11f8de30cb3c1fde9632d161e1573e0f4a9d1e8bda920ab217302eb3e8d\": rpc error: code = NotFound desc = could not find container \"b7cbe11f8de30cb3c1fde9632d161e1573e0f4a9d1e8bda920ab217302eb3e8d\": container with ID starting with b7cbe11f8de30cb3c1fde9632d161e1573e0f4a9d1e8bda920ab217302eb3e8d not found: ID does not exist" Sep 19 06:09:23.741388 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:23.741357 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:09:24.243468 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:24.243444 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8102-13da09" Sep 19 06:09:24.267538 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:24.267504 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50c0a89d-a1f8-41b4-ac80-543b46fece8b" path="/var/lib/kubelet/pods/50c0a89d-a1f8-41b4-ac80-543b46fece8b/volumes" Sep 19 06:09:24.294162 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:24.291223 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwjss\" (UniqueName: \"kubernetes.io/projected/cd5c484e-1e34-4cf7-a039-36f964438006-kube-api-access-qwjss\") pod \"cd5c484e-1e34-4cf7-a039-36f964438006\" (UID: \"cd5c484e-1e34-4cf7-a039-36f964438006\") " Sep 19 06:09:24.296448 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:24.294763 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd5c484e-1e34-4cf7-a039-36f964438006-kube-api-access-qwjss" (OuterVolumeSpecName: "kube-api-access-qwjss") pod "cd5c484e-1e34-4cf7-a039-36f964438006" (UID: "cd5c484e-1e34-4cf7-a039-36f964438006"). InnerVolumeSpecName "kube-api-access-qwjss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:09:24.393597 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:24.393552 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwjss\" (UniqueName: \"kubernetes.io/projected/cd5c484e-1e34-4cf7-a039-36f964438006-kube-api-access-qwjss\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:09:25.087975 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:25.087933 2584 generic.go:356] "Generic (PLEG): container finished" podID="cd5c484e-1e34-4cf7-a039-36f964438006" containerID="9214f0844be4f19233a703544d3478949ea6e2c15444f2aa8defe3f672095ac4" exitCode=137 Sep 19 06:09:25.088158 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:25.087985 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8102-13da09" Sep 19 06:09:25.088158 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:25.088032 2584 scope.go:117] "RemoveContainer" containerID="9214f0844be4f19233a703544d3478949ea6e2c15444f2aa8defe3f672095ac4" Sep 19 06:09:25.097296 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:25.097277 2584 scope.go:117] "RemoveContainer" containerID="9214f0844be4f19233a703544d3478949ea6e2c15444f2aa8defe3f672095ac4" Sep 19 06:09:25.097649 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:09:25.097625 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9214f0844be4f19233a703544d3478949ea6e2c15444f2aa8defe3f672095ac4\": container with ID starting with 9214f0844be4f19233a703544d3478949ea6e2c15444f2aa8defe3f672095ac4 not found: ID does not exist" containerID="9214f0844be4f19233a703544d3478949ea6e2c15444f2aa8defe3f672095ac4" Sep 19 06:09:25.097750 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:25.097654 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9214f0844be4f19233a703544d3478949ea6e2c15444f2aa8defe3f672095ac4"} err="failed to get container status \"9214f0844be4f19233a703544d3478949ea6e2c15444f2aa8defe3f672095ac4\": rpc error: code = NotFound desc = could not find container \"9214f0844be4f19233a703544d3478949ea6e2c15444f2aa8defe3f672095ac4\": container with ID starting with 9214f0844be4f19233a703544d3478949ea6e2c15444f2aa8defe3f672095ac4 not found: ID does not exist" Sep 19 06:09:26.266789 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:26.266758 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd5c484e-1e34-4cf7-a039-36f964438006" path="/var/lib/kubelet/pods/cd5c484e-1e34-4cf7-a039-36f964438006/volumes" Sep 19 06:09:32.172368 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:32.172334 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:09:53.181206 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:09:53.181168 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:10:12.447928 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:10:12.447858 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:10:12.500825 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:10:12.500792 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:10:21.050530 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:10:21.050494 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:00.726481 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:00.726368 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:05.468617 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:05.468583 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:06.469163 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:06.469132 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:06.916633 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:06.916594 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:07.212261 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.212178 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n"] Sep 19 06:11:07.212785 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.212765 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd5c484e-1e34-4cf7-a039-36f964438006" containerName="test-curl-8102-13da09" Sep 19 06:11:07.212841 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.212788 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd5c484e-1e34-4cf7-a039-36f964438006" containerName="test-curl-8102-13da09" Sep 19 06:11:07.212884 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.212871 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd5c484e-1e34-4cf7-a039-36f964438006" containerName="test-curl-8102-13da09" Sep 19 06:11:07.215616 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.215597 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.218053 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.218028 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 19 06:11:07.218216 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.218136 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-7142e3-openshift-default-dockercfg-6h2zh\"" Sep 19 06:11:07.225533 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.225496 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n"] Sep 19 06:11:07.235856 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.235827 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:07.294121 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.294073 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-credential-socket\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.294319 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.294225 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istiod-ca-cert\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.294387 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.294373 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-workload-socket\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.294453 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.294438 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-podinfo\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.294534 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.294518 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-token\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.294571 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.294549 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-workload-certs\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.294609 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.294585 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-data\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.294609 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.294603 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-envoy\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.294674 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.294621 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snmz9\" (UniqueName: \"kubernetes.io/projected/9059cab0-dc5c-4595-a718-18e2ffd10bb1-kube-api-access-snmz9\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396008 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.395962 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-credential-socket\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396196 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396021 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istiod-ca-cert\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396196 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396063 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-workload-socket\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396196 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396085 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-podinfo\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396196 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396141 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-token\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396196 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396171 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-workload-certs\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396524 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396204 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-data\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396524 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396233 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-envoy\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396524 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396256 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmz9\" (UniqueName: \"kubernetes.io/projected/9059cab0-dc5c-4595-a718-18e2ffd10bb1-kube-api-access-snmz9\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396524 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396470 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-credential-socket\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396524 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396520 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-workload-socket\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396748 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396619 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-workload-certs\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396786 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396741 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-data\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.396874 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.396856 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istiod-ca-cert\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.398675 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.398639 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-envoy\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.398882 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.398863 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-podinfo\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.403566 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.403543 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-token\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.403704 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.403625 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snmz9\" (UniqueName: \"kubernetes.io/projected/9059cab0-dc5c-4595-a718-18e2ffd10bb1-kube-api-access-snmz9\") pod \"e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.526735 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.526652 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:07.528235 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.528211 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:07.801093 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.801065 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n"] Sep 19 06:11:07.802903 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:11:07.802871 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9059cab0_dc5c_4595_a718_18e2ffd10bb1.slice/crio-f2525c447c2b1a96b86002da4132dd0c18b57ab5fc83393acb5b2d04e96dad5b WatchSource:0}: Error finding container f2525c447c2b1a96b86002da4132dd0c18b57ab5fc83393acb5b2d04e96dad5b: Status 404 returned error can't find the container with id f2525c447c2b1a96b86002da4132dd0c18b57ab5fc83393acb5b2d04e96dad5b Sep 19 06:11:07.805067 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.805033 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:11:07.805139 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.805104 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:11:07.805139 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:07.805132 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:11:08.660235 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:08.660188 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" event={"ID":"9059cab0-dc5c-4595-a718-18e2ffd10bb1","Type":"ContainerStarted","Data":"219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e"} Sep 19 06:11:08.660689 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:08.660255 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" event={"ID":"9059cab0-dc5c-4595-a718-18e2ffd10bb1","Type":"ContainerStarted","Data":"f2525c447c2b1a96b86002da4132dd0c18b57ab5fc83393acb5b2d04e96dad5b"} Sep 19 06:11:08.679227 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:08.679174 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" podStartSLOduration=1.679157139 podStartE2EDuration="1.679157139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:11:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:11:08.677689796 +0000 UTC m=+1796.670917326" watchObservedRunningTime="2026-09-19 06:11:08.679157139 +0000 UTC m=+1796.672384670" Sep 19 06:11:09.528506 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:09.528467 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:09.533894 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:09.533857 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:09.665659 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:09.665629 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:09.666790 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:09.666768 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:09.980896 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:09.980827 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:10.578594 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:10.578542 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:10.679550 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:10.679515 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:10.779469 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:10.779428 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:11.430965 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:11.430923 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:12.100980 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.100950 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:12.152207 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.152174 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:12.272948 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.272910 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:12.583760 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.583718 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:12.681312 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.681275 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:12.693637 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.693605 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74"] Sep 19 06:11:12.697414 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.697374 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" Sep 19 06:11:12.700099 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.700076 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-7142e3\"" Sep 19 06:11:12.700254 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.700123 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-7142e3-dockercfg-2bn8b\"" Sep 19 06:11:12.717122 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.717089 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74"] Sep 19 06:11:12.746273 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.746233 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdfq2\" (UniqueName: \"kubernetes.io/projected/6f8242e6-6f42-4da2-a95e-886c3587b3eb-kube-api-access-vdfq2\") pod \"payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74\" (UID: \"6f8242e6-6f42-4da2-a95e-886c3587b3eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" Sep 19 06:11:12.746494 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.746431 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f8242e6-6f42-4da2-a95e-886c3587b3eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74\" (UID: \"6f8242e6-6f42-4da2-a95e-886c3587b3eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" Sep 19 06:11:12.847698 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.847594 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdfq2\" (UniqueName: \"kubernetes.io/projected/6f8242e6-6f42-4da2-a95e-886c3587b3eb-kube-api-access-vdfq2\") pod \"payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74\" (UID: \"6f8242e6-6f42-4da2-a95e-886c3587b3eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" Sep 19 06:11:12.847698 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.847636 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f8242e6-6f42-4da2-a95e-886c3587b3eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74\" (UID: \"6f8242e6-6f42-4da2-a95e-886c3587b3eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" Sep 19 06:11:12.848281 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.848258 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f8242e6-6f42-4da2-a95e-886c3587b3eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74\" (UID: \"6f8242e6-6f42-4da2-a95e-886c3587b3eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" Sep 19 06:11:12.856067 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:12.856041 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdfq2\" (UniqueName: \"kubernetes.io/projected/6f8242e6-6f42-4da2-a95e-886c3587b3eb-kube-api-access-vdfq2\") pod \"payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74\" (UID: \"6f8242e6-6f42-4da2-a95e-886c3587b3eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" Sep 19 06:11:13.008793 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:13.008749 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" Sep 19 06:11:13.282743 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:13.282710 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74"] Sep 19 06:11:13.285245 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:11:13.285202 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f8242e6_6f42_4da2_a95e_886c3587b3eb.slice/crio-67182c34b844bba885cd1e8ce4b9e13d213cfca4874b85f388263b9ab82ac3e7 WatchSource:0}: Error finding container 67182c34b844bba885cd1e8ce4b9e13d213cfca4874b85f388263b9ab82ac3e7: Status 404 returned error can't find the container with id 67182c34b844bba885cd1e8ce4b9e13d213cfca4874b85f388263b9ab82ac3e7 Sep 19 06:11:13.689178 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:13.689140 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" event={"ID":"6f8242e6-6f42-4da2-a95e-886c3587b3eb","Type":"ContainerStarted","Data":"8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab"} Sep 19 06:11:13.689360 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:13.689207 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" event={"ID":"6f8242e6-6f42-4da2-a95e-886c3587b3eb","Type":"ContainerStarted","Data":"67182c34b844bba885cd1e8ce4b9e13d213cfca4874b85f388263b9ab82ac3e7"} Sep 19 06:11:13.689360 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:13.689239 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" Sep 19 06:11:13.704079 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:13.704023 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" podStartSLOduration=1.704003818 podStartE2EDuration="1.704003818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:11:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:11:13.702840645 +0000 UTC m=+1801.696068171" watchObservedRunningTime="2026-09-19 06:11:13.704003818 +0000 UTC m=+1801.697231349" Sep 19 06:11:14.383931 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:14.383905 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 06:11:14.387479 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:14.387452 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 06:11:24.696830 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:24.696800 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" Sep 19 06:11:25.706627 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:25.706569 2584 patch_prober.go:28] interesting pod/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 19 06:11:25.707157 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:25.706671 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 19 06:11:27.781990 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:27.781887 2584 generic.go:356] "Generic (PLEG): container finished" podID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerID="219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e" exitCode=0 Sep 19 06:11:27.781990 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:27.781930 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" event={"ID":"9059cab0-dc5c-4595-a718-18e2ffd10bb1","Type":"ContainerDied","Data":"219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e"} Sep 19 06:11:27.782458 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:27.782354 2584 scope.go:117] "RemoveContainer" containerID="219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e" Sep 19 06:11:27.783507 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:27.783473 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:11:27.783637 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:27.783561 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:11:27.783637 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:27.783611 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 19 06:11:28.789880 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:28.789837 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" event={"ID":"9059cab0-dc5c-4595-a718-18e2ffd10bb1","Type":"ContainerStarted","Data":"e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37"} Sep 19 06:11:29.529527 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:29.529487 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:29.529958 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:29.529852 2584 patch_prober.go:28] interesting pod/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body= Sep 19 06:11:29.530173 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:29.530141 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" Sep 19 06:11:30.529069 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:30.529025 2584 patch_prober.go:28] interesting pod/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body= Sep 19 06:11:30.530124 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:30.529681 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" Sep 19 06:11:31.530127 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:31.529653 2584 patch_prober.go:28] interesting pod/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body= Sep 19 06:11:31.530127 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:31.529722 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" Sep 19 06:11:32.529186 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:32.529132 2584 patch_prober.go:28] interesting pod/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body= Sep 19 06:11:32.529449 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:32.529209 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" Sep 19 06:11:33.529616 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:33.529120 2584 patch_prober.go:28] interesting pod/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body= Sep 19 06:11:33.529616 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:33.529193 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" Sep 19 06:11:34.483930 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:34.483892 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:34.532068 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:34.532036 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:34.532486 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:34.532321 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:34.533097 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:34.533078 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:11:35.137351 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:35.137304 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:35.599213 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:35.599180 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:35.904818 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:35.904724 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:36.309952 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:36.309858 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:36.610662 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:36.610633 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:36.710306 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:36.710271 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:36.981136 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:36.981056 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:38.999206 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:38.999167 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:39.792487 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:39.792447 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:39.891120 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:39.891089 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:40.728635 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:40.728602 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:41.697635 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:41.697604 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:42.365152 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:42.365110 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:42.466266 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:42.466235 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:42.883559 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:42.883510 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:42.985689 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:42.985652 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:43.011466 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.011425 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp"] Sep 19 06:11:43.013973 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.013952 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" Sep 19 06:11:43.016225 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.016200 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-ebb0f4-dockercfg-s8pkk\"" Sep 19 06:11:43.016674 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.016544 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-ebb0f4\"" Sep 19 06:11:43.028788 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.028763 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp"] Sep 19 06:11:43.050427 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.050376 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh"] Sep 19 06:11:43.053618 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.053591 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" Sep 19 06:11:43.061798 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.061767 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh"] Sep 19 06:11:43.063746 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.063681 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:43.076680 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.076641 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwqqk\" (UniqueName: \"kubernetes.io/projected/525145f8-7541-48c2-82b8-0b168e17cc25-kube-api-access-wwqqk\") pod \"payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp\" (UID: \"525145f8-7541-48c2-82b8-0b168e17cc25\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" Sep 19 06:11:43.076980 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.076960 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/525145f8-7541-48c2-82b8-0b168e17cc25-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp\" (UID: \"525145f8-7541-48c2-82b8-0b168e17cc25\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" Sep 19 06:11:43.178494 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.178378 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8f87f58-b4da-4270-90fb-219b04573ed7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh\" (UID: \"b8f87f58-b4da-4270-90fb-219b04573ed7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" Sep 19 06:11:43.178651 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.178536 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxsp5\" (UniqueName: \"kubernetes.io/projected/b8f87f58-b4da-4270-90fb-219b04573ed7-kube-api-access-fxsp5\") pod \"payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh\" (UID: \"b8f87f58-b4da-4270-90fb-219b04573ed7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" Sep 19 06:11:43.178651 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.178583 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwqqk\" (UniqueName: \"kubernetes.io/projected/525145f8-7541-48c2-82b8-0b168e17cc25-kube-api-access-wwqqk\") pod \"payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp\" (UID: \"525145f8-7541-48c2-82b8-0b168e17cc25\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" Sep 19 06:11:43.178734 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.178703 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/525145f8-7541-48c2-82b8-0b168e17cc25-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp\" (UID: \"525145f8-7541-48c2-82b8-0b168e17cc25\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" Sep 19 06:11:43.179680 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.179654 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/525145f8-7541-48c2-82b8-0b168e17cc25-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp\" (UID: \"525145f8-7541-48c2-82b8-0b168e17cc25\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" Sep 19 06:11:43.186504 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.186477 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwqqk\" (UniqueName: \"kubernetes.io/projected/525145f8-7541-48c2-82b8-0b168e17cc25-kube-api-access-wwqqk\") pod \"payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp\" (UID: \"525145f8-7541-48c2-82b8-0b168e17cc25\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" Sep 19 06:11:43.279845 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.279800 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8f87f58-b4da-4270-90fb-219b04573ed7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh\" (UID: \"b8f87f58-b4da-4270-90fb-219b04573ed7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" Sep 19 06:11:43.280063 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.279884 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxsp5\" (UniqueName: \"kubernetes.io/projected/b8f87f58-b4da-4270-90fb-219b04573ed7-kube-api-access-fxsp5\") pod \"payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh\" (UID: \"b8f87f58-b4da-4270-90fb-219b04573ed7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" Sep 19 06:11:43.280585 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.280561 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8f87f58-b4da-4270-90fb-219b04573ed7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh\" (UID: \"b8f87f58-b4da-4270-90fb-219b04573ed7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" Sep 19 06:11:43.287539 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.287511 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxsp5\" (UniqueName: \"kubernetes.io/projected/b8f87f58-b4da-4270-90fb-219b04573ed7-kube-api-access-fxsp5\") pod \"payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh\" (UID: \"b8f87f58-b4da-4270-90fb-219b04573ed7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" Sep 19 06:11:43.326355 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.326314 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" Sep 19 06:11:43.369436 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.369388 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" Sep 19 06:11:43.607136 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.607101 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp"] Sep 19 06:11:43.608794 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:11:43.608768 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod525145f8_7541_48c2_82b8_0b168e17cc25.slice/crio-47724e3c8deb6f462bed403027d159e2d7fa8dc42c6f76f51df1faab2b06a1c1 WatchSource:0}: Error finding container 47724e3c8deb6f462bed403027d159e2d7fa8dc42c6f76f51df1faab2b06a1c1: Status 404 returned error can't find the container with id 47724e3c8deb6f462bed403027d159e2d7fa8dc42c6f76f51df1faab2b06a1c1 Sep 19 06:11:43.650858 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.650830 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh"] Sep 19 06:11:43.653639 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:11:43.653613 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8f87f58_b4da_4270_90fb_219b04573ed7.slice/crio-2658d2b848e5f604d25988841d54295d3fd4b3cc90c3daf0c216bdd076a0cc7f WatchSource:0}: Error finding container 2658d2b848e5f604d25988841d54295d3fd4b3cc90c3daf0c216bdd076a0cc7f: Status 404 returned error can't find the container with id 2658d2b848e5f604d25988841d54295d3fd4b3cc90c3daf0c216bdd076a0cc7f Sep 19 06:11:43.887018 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.886962 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" event={"ID":"525145f8-7541-48c2-82b8-0b168e17cc25","Type":"ContainerStarted","Data":"339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e"} Sep 19 06:11:43.887235 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.887026 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" event={"ID":"525145f8-7541-48c2-82b8-0b168e17cc25","Type":"ContainerStarted","Data":"47724e3c8deb6f462bed403027d159e2d7fa8dc42c6f76f51df1faab2b06a1c1"} Sep 19 06:11:43.887235 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.887130 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" Sep 19 06:11:43.888725 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.888701 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" event={"ID":"b8f87f58-b4da-4270-90fb-219b04573ed7","Type":"ContainerStarted","Data":"63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137"} Sep 19 06:11:43.888854 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.888738 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" event={"ID":"b8f87f58-b4da-4270-90fb-219b04573ed7","Type":"ContainerStarted","Data":"2658d2b848e5f604d25988841d54295d3fd4b3cc90c3daf0c216bdd076a0cc7f"} Sep 19 06:11:43.888854 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.888758 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" Sep 19 06:11:43.903178 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.903122 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" podStartSLOduration=1.903102528 podStartE2EDuration="1.903102528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:11:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:11:43.901605189 +0000 UTC m=+1831.894832713" watchObservedRunningTime="2026-09-19 06:11:43.903102528 +0000 UTC m=+1831.896330058" Sep 19 06:11:43.918911 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:43.918840 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" podStartSLOduration=0.91881721 podStartE2EDuration="918.81721ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:11:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:11:43.915708238 +0000 UTC m=+1831.908935770" watchObservedRunningTime="2026-09-19 06:11:43.91881721 +0000 UTC m=+1831.912044745" Sep 19 06:11:52.168839 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:52.168802 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:53.876040 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:53.875991 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:53.922055 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:53.922009 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:11:54.897655 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:54.897620 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" Sep 19 06:11:54.898145 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:11:54.897703 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" Sep 19 06:12:03.982118 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:03.982062 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:05.179568 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:05.179512 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:05.972232 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:05.972196 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:06.590824 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:06.590784 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:07.217722 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:07.217692 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:07.991692 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:07.991658 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:09.841707 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:09.841672 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:14.981577 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:14.981524 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dmnts_e3c2b6d2-b60c-4dde-906b-7301fea6c7f5/payload-processing/0.log" Sep 19 06:12:15.099984 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:15.099953 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh_b8f87f58-b4da-4270-90fb-219b04573ed7/payload-processing/0.log" Sep 19 06:12:17.714387 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:17.714351 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:24.774120 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:24.774087 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:51.279341 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:51.279255 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dmnts_e3c2b6d2-b60c-4dde-906b-7301fea6c7f5/payload-processing/0.log" Sep 19 06:12:51.395796 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:51.395763 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh_b8f87f58-b4da-4270-90fb-219b04573ed7/payload-processing/0.log" Sep 19 06:12:51.511194 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:51.511167 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh_b8f87f58-b4da-4270-90fb-219b04573ed7/payload-processing/0.log" Sep 19 06:12:51.786604 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:51.786566 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:53.633555 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:53.633513 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:54.607969 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:54.607922 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:55.218295 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:55.218256 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:55.321004 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:55.320975 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:55.422156 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:55.422122 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:56.085629 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:56.085594 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:56.719630 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:56.719592 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:57.389943 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:57.389912 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:58.026951 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.026920 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:58.082654 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.082621 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:58.497757 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.497714 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:58.569318 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.569251 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj"] Sep 19 06:12:58.572002 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.571978 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:58.573720 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.573703 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" Sep 19 06:12:58.576184 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.576159 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-4eb285\"" Sep 19 06:12:58.576184 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.576159 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-4eb285-dockercfg-pshxd\"" Sep 19 06:12:58.584973 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.584942 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj"] Sep 19 06:12:58.641766 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.641730 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:12:58.654529 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.654500 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xd6b\" (UniqueName: \"kubernetes.io/projected/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06-kube-api-access-9xd6b\") pod \"payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj\" (UID: \"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" Sep 19 06:12:58.654654 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.654574 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj\" (UID: \"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" Sep 19 06:12:58.756101 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.755989 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xd6b\" (UniqueName: \"kubernetes.io/projected/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06-kube-api-access-9xd6b\") pod \"payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj\" (UID: \"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" Sep 19 06:12:58.756101 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.756089 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj\" (UID: \"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" Sep 19 06:12:58.756917 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.756889 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj\" (UID: \"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" Sep 19 06:12:58.763567 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.763546 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xd6b\" (UniqueName: \"kubernetes.io/projected/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06-kube-api-access-9xd6b\") pod \"payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj\" (UID: \"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" Sep 19 06:12:58.886041 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:58.886001 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" Sep 19 06:12:59.158219 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:59.158188 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj"] Sep 19 06:12:59.159970 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:12:59.159942 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cd67438_e1d6_45fd_b7cc_9aa3ad199b06.slice/crio-ab52fd19375b621a9720036438b56a73accf6b9d4b20456496f47c6c2750f705 WatchSource:0}: Error finding container ab52fd19375b621a9720036438b56a73accf6b9d4b20456496f47c6c2750f705: Status 404 returned error can't find the container with id ab52fd19375b621a9720036438b56a73accf6b9d4b20456496f47c6c2750f705 Sep 19 06:12:59.338824 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:59.338787 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" event={"ID":"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06","Type":"ContainerStarted","Data":"cc7301f74549adb81d7a1467a9b6557ffc329d5d14c84914cde733abf5a88b4c"} Sep 19 06:12:59.339035 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:59.338847 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" event={"ID":"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06","Type":"ContainerStarted","Data":"ab52fd19375b621a9720036438b56a73accf6b9d4b20456496f47c6c2750f705"} Sep 19 06:12:59.339035 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:59.338879 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" Sep 19 06:12:59.354627 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:12:59.354569 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" podStartSLOduration=1.354554053 podStartE2EDuration="1.354554053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:12:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:12:59.35313421 +0000 UTC m=+1907.346361766" watchObservedRunningTime="2026-09-19 06:12:59.354554053 +0000 UTC m=+1907.347781584" Sep 19 06:13:10.346603 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:10.346567 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" Sep 19 06:13:14.781376 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:14.781338 2584 scope.go:117] "RemoveContainer" containerID="cc3106c58ef9f6d94c2a00efe19eee105652322a06197237e0928d5c69d0ce63" Sep 19 06:13:14.787694 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:14.787668 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:14.792280 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:14.792254 2584 scope.go:117] "RemoveContainer" containerID="947a4af26b3e6e993628f57743fcaa6d0ab3454b519722e80bf4aeaea4a846da" Sep 19 06:13:14.802480 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:14.802460 2584 scope.go:117] "RemoveContainer" containerID="69b919ad0986c7ed56aa4b68da2ee0550a176ebc6532eba6e8c9d5bcf966727e" Sep 19 06:13:14.811790 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:14.811764 2584 scope.go:117] "RemoveContainer" containerID="0ebce582c6bc97d4daa64e02f2b6639d3687a6b037d908662302ff509fa09df1" Sep 19 06:13:20.301913 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:20.301878 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:20.573329 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:20.573239 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:20.976563 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:20.976511 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds"] Sep 19 06:13:20.981919 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:20.981895 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds" Sep 19 06:13:20.984700 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:20.984672 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:13:20.986753 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:20.986727 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds"] Sep 19 06:13:21.085485 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:21.085451 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c30bd39-e71b-4959-b11a-3e7466eac26f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds\" (UID: \"2c30bd39-e71b-4959-b11a-3e7466eac26f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds" Sep 19 06:13:21.186650 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:21.186613 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c30bd39-e71b-4959-b11a-3e7466eac26f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds\" (UID: \"2c30bd39-e71b-4959-b11a-3e7466eac26f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds" Sep 19 06:13:21.187259 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:21.187233 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c30bd39-e71b-4959-b11a-3e7466eac26f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds\" (UID: \"2c30bd39-e71b-4959-b11a-3e7466eac26f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds" Sep 19 06:13:21.296075 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:21.295991 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds" Sep 19 06:13:21.483513 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:21.483477 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:21.569807 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:21.569779 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds"] Sep 19 06:13:21.571991 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:13:21.571960 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c30bd39_e71b_4959_b11a_3e7466eac26f.slice/crio-03dac2238e3e406760b1725d0f4cefe147b196addac20044781585fe336408b7 WatchSource:0}: Error finding container 03dac2238e3e406760b1725d0f4cefe147b196addac20044781585fe336408b7: Status 404 returned error can't find the container with id 03dac2238e3e406760b1725d0f4cefe147b196addac20044781585fe336408b7 Sep 19 06:13:22.209649 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:22.209611 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:22.473917 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:22.473817 2584 generic.go:356] "Generic (PLEG): container finished" podID="2c30bd39-e71b-4959-b11a-3e7466eac26f" containerID="1ea9f051c350e095426bc6809860690b13ea5ff709fd7e58e98d4e4b1ddb473b" exitCode=0 Sep 19 06:13:22.473917 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:22.473869 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds" event={"ID":"2c30bd39-e71b-4959-b11a-3e7466eac26f","Type":"ContainerDied","Data":"1ea9f051c350e095426bc6809860690b13ea5ff709fd7e58e98d4e4b1ddb473b"} Sep 19 06:13:22.474111 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:22.473928 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds" event={"ID":"2c30bd39-e71b-4959-b11a-3e7466eac26f","Type":"ContainerStarted","Data":"03dac2238e3e406760b1725d0f4cefe147b196addac20044781585fe336408b7"} Sep 19 06:13:22.858475 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:22.858445 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:23.587501 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:23.587467 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:23.615056 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:23.615029 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds" Sep 19 06:13:23.712501 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:23.712460 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c30bd39-e71b-4959-b11a-3e7466eac26f-maas-api-service-ca\") pod \"2c30bd39-e71b-4959-b11a-3e7466eac26f\" (UID: \"2c30bd39-e71b-4959-b11a-3e7466eac26f\") " Sep 19 06:13:23.712849 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:23.712824 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c30bd39-e71b-4959-b11a-3e7466eac26f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c30bd39-e71b-4959-b11a-3e7466eac26f" (UID: "2c30bd39-e71b-4959-b11a-3e7466eac26f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:13:23.712964 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:23.712945 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c30bd39-e71b-4959-b11a-3e7466eac26f-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:13:24.215836 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:24.215803 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:24.486685 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:24.486593 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds" Sep 19 06:13:24.486685 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:24.486589 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds" event={"ID":"2c30bd39-e71b-4959-b11a-3e7466eac26f","Type":"ContainerDied","Data":"03dac2238e3e406760b1725d0f4cefe147b196addac20044781585fe336408b7"} Sep 19 06:13:24.486685 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:24.486725 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03dac2238e3e406760b1725d0f4cefe147b196addac20044781585fe336408b7" Sep 19 06:13:24.857496 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:24.857462 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:26.790384 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:26.790343 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:27.320367 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.320327 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:27.355049 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.355009 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj"] Sep 19 06:13:27.355299 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.355271 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" podUID="3cd67438-e1d6-45fd-b7cc-9aa3ad199b06" containerName="payload-pre-processing" containerID="cri-o://cc7301f74549adb81d7a1467a9b6557ffc329d5d14c84914cde733abf5a88b4c" gracePeriod=30 Sep 19 06:13:27.383860 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.383835 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:27.483684 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.483647 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:27.494957 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.494893 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds"] Sep 19 06:13:27.497253 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.497224 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4eb285-z56ds"] Sep 19 06:13:27.509491 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.509460 2584 generic.go:356] "Generic (PLEG): container finished" podID="3cd67438-e1d6-45fd-b7cc-9aa3ad199b06" containerID="cc7301f74549adb81d7a1467a9b6557ffc329d5d14c84914cde733abf5a88b4c" exitCode=0 Sep 19 06:13:27.509654 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.509542 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" event={"ID":"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06","Type":"ContainerDied","Data":"cc7301f74549adb81d7a1467a9b6557ffc329d5d14c84914cde733abf5a88b4c"} Sep 19 06:13:27.569269 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.569230 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:27.615830 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.615797 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" Sep 19 06:13:27.652551 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.652511 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xd6b\" (UniqueName: \"kubernetes.io/projected/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06-kube-api-access-9xd6b\") pod \"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06\" (UID: \"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06\") " Sep 19 06:13:27.652735 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.652592 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06-plugins-config-volume\") pod \"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06\" (UID: \"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06\") " Sep 19 06:13:27.652956 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.652928 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3cd67438-e1d6-45fd-b7cc-9aa3ad199b06" (UID: "3cd67438-e1d6-45fd-b7cc-9aa3ad199b06"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:13:27.654777 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.654749 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06-kube-api-access-9xd6b" (OuterVolumeSpecName: "kube-api-access-9xd6b") pod "3cd67438-e1d6-45fd-b7cc-9aa3ad199b06" (UID: "3cd67438-e1d6-45fd-b7cc-9aa3ad199b06"). InnerVolumeSpecName "kube-api-access-9xd6b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:13:27.753710 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.753675 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:13:27.753710 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.753707 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xd6b\" (UniqueName: \"kubernetes.io/projected/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06-kube-api-access-9xd6b\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:13:27.858590 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.858497 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:27.995754 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:27.995716 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:28.269508 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:28.269420 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c30bd39-e71b-4959-b11a-3e7466eac26f" path="/var/lib/kubelet/pods/2c30bd39-e71b-4959-b11a-3e7466eac26f/volumes" Sep 19 06:13:28.516555 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:28.516520 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" Sep 19 06:13:28.516726 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:28.516520 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj" event={"ID":"3cd67438-e1d6-45fd-b7cc-9aa3ad199b06","Type":"ContainerDied","Data":"ab52fd19375b621a9720036438b56a73accf6b9d4b20456496f47c6c2750f705"} Sep 19 06:13:28.516726 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:28.516649 2584 scope.go:117] "RemoveContainer" containerID="cc7301f74549adb81d7a1467a9b6557ffc329d5d14c84914cde733abf5a88b4c" Sep 19 06:13:28.532771 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:28.532740 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj"] Sep 19 06:13:28.536325 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:28.536293 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4eb285-79bd4d5c98-zfddj"] Sep 19 06:13:28.689682 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:28.689653 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:29.314600 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:29.314568 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:30.093500 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:30.093467 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:30.267837 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:30.267801 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cd67438-e1d6-45fd-b7cc-9aa3ad199b06" path="/var/lib/kubelet/pods/3cd67438-e1d6-45fd-b7cc-9aa3ad199b06/volumes" Sep 19 06:13:30.709696 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:30.709662 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:33.390344 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:33.390299 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:33.490152 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:33.490123 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:33.662686 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:33.662604 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:37.373679 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.373640 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m"] Sep 19 06:13:37.374114 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.374101 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c30bd39-e71b-4959-b11a-3e7466eac26f" containerName="revoke-keys" Sep 19 06:13:37.374114 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.374116 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c30bd39-e71b-4959-b11a-3e7466eac26f" containerName="revoke-keys" Sep 19 06:13:37.374190 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.374145 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cd67438-e1d6-45fd-b7cc-9aa3ad199b06" containerName="payload-pre-processing" Sep 19 06:13:37.374190 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.374151 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cd67438-e1d6-45fd-b7cc-9aa3ad199b06" containerName="payload-pre-processing" Sep 19 06:13:37.374254 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.374215 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c30bd39-e71b-4959-b11a-3e7466eac26f" containerName="revoke-keys" Sep 19 06:13:37.374254 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.374225 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cd67438-e1d6-45fd-b7cc-9aa3ad199b06" containerName="payload-pre-processing" Sep 19 06:13:37.377393 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.377375 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m" Sep 19 06:13:37.379526 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.379501 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:13:37.384004 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.383979 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m"] Sep 19 06:13:37.455562 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.455523 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba0412ac-8296-4e11-a3aa-65baf26ab2df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m\" (UID: \"ba0412ac-8296-4e11-a3aa-65baf26ab2df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m" Sep 19 06:13:37.556578 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.556527 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba0412ac-8296-4e11-a3aa-65baf26ab2df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m\" (UID: \"ba0412ac-8296-4e11-a3aa-65baf26ab2df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m" Sep 19 06:13:37.557205 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.557182 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba0412ac-8296-4e11-a3aa-65baf26ab2df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m\" (UID: \"ba0412ac-8296-4e11-a3aa-65baf26ab2df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m" Sep 19 06:13:37.689793 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.689695 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m" Sep 19 06:13:37.692139 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.692110 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:37.961301 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:37.961217 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m"] Sep 19 06:13:37.964629 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:13:37.964600 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba0412ac_8296_4e11_a3aa_65baf26ab2df.slice/crio-19f1e01846c9819d5d6c6aa05d1a464e09f39cdf822576f4f467989cab31c594 WatchSource:0}: Error finding container 19f1e01846c9819d5d6c6aa05d1a464e09f39cdf822576f4f467989cab31c594: Status 404 returned error can't find the container with id 19f1e01846c9819d5d6c6aa05d1a464e09f39cdf822576f4f467989cab31c594 Sep 19 06:13:38.381363 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:38.381332 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:38.574049 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:38.573953 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m" event={"ID":"ba0412ac-8296-4e11-a3aa-65baf26ab2df","Type":"ContainerStarted","Data":"8183e36d1de5572b359809fe9280da72c0a7a37c56fbf26a7045a9f573d36975"} Sep 19 06:13:38.574049 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:38.574008 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m" event={"ID":"ba0412ac-8296-4e11-a3aa-65baf26ab2df","Type":"ContainerStarted","Data":"19f1e01846c9819d5d6c6aa05d1a464e09f39cdf822576f4f467989cab31c594"} Sep 19 06:13:38.587481 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:38.587431 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m" podStartSLOduration=1.587391781 podStartE2EDuration="1.587391781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:13:38.587012967 +0000 UTC m=+1946.580240498" watchObservedRunningTime="2026-09-19 06:13:38.587391781 +0000 UTC m=+1946.580619313" Sep 19 06:13:38.999691 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:38.999659 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:39.580909 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:39.580872 2584 generic.go:356] "Generic (PLEG): container finished" podID="ba0412ac-8296-4e11-a3aa-65baf26ab2df" containerID="8183e36d1de5572b359809fe9280da72c0a7a37c56fbf26a7045a9f573d36975" exitCode=0 Sep 19 06:13:39.581271 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:39.580959 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m" event={"ID":"ba0412ac-8296-4e11-a3aa-65baf26ab2df","Type":"ContainerDied","Data":"8183e36d1de5572b359809fe9280da72c0a7a37c56fbf26a7045a9f573d36975"} Sep 19 06:13:39.681350 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:39.681314 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:40.406256 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:40.406207 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:40.726343 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:40.726312 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m" Sep 19 06:13:40.793029 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:40.792997 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba0412ac-8296-4e11-a3aa-65baf26ab2df-maas-api-service-ca\") pod \"ba0412ac-8296-4e11-a3aa-65baf26ab2df\" (UID: \"ba0412ac-8296-4e11-a3aa-65baf26ab2df\") " Sep 19 06:13:40.793354 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:40.793330 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba0412ac-8296-4e11-a3aa-65baf26ab2df-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ba0412ac-8296-4e11-a3aa-65baf26ab2df" (UID: "ba0412ac-8296-4e11-a3aa-65baf26ab2df"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:13:40.894272 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:40.894232 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba0412ac-8296-4e11-a3aa-65baf26ab2df-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:13:41.182843 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:41.182811 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:41.595412 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:41.595372 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m" event={"ID":"ba0412ac-8296-4e11-a3aa-65baf26ab2df","Type":"ContainerDied","Data":"19f1e01846c9819d5d6c6aa05d1a464e09f39cdf822576f4f467989cab31c594"} Sep 19 06:13:41.595592 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:41.595440 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19f1e01846c9819d5d6c6aa05d1a464e09f39cdf822576f4f467989cab31c594" Sep 19 06:13:41.595592 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:41.595390 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m" Sep 19 06:13:52.899812 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:52.899771 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:52.938093 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:52.938054 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74"] Sep 19 06:13:52.938369 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:52.938319 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" podUID="6f8242e6-6f42-4da2-a95e-886c3587b3eb" containerName="payload-pre-processing" containerID="cri-o://8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab" gracePeriod=30 Sep 19 06:13:53.000038 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.000009 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:53.090073 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.090032 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m"] Sep 19 06:13:53.092180 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.092115 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:53.092460 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.092437 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7142e3-5xv5m"] Sep 19 06:13:53.144504 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.144363 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:53.200670 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.200644 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" Sep 19 06:13:53.322178 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.322144 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdfq2\" (UniqueName: \"kubernetes.io/projected/6f8242e6-6f42-4da2-a95e-886c3587b3eb-kube-api-access-vdfq2\") pod \"6f8242e6-6f42-4da2-a95e-886c3587b3eb\" (UID: \"6f8242e6-6f42-4da2-a95e-886c3587b3eb\") " Sep 19 06:13:53.322382 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.322303 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f8242e6-6f42-4da2-a95e-886c3587b3eb-plugins-config-volume\") pod \"6f8242e6-6f42-4da2-a95e-886c3587b3eb\" (UID: \"6f8242e6-6f42-4da2-a95e-886c3587b3eb\") " Sep 19 06:13:53.322728 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.322695 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f8242e6-6f42-4da2-a95e-886c3587b3eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6f8242e6-6f42-4da2-a95e-886c3587b3eb" (UID: "6f8242e6-6f42-4da2-a95e-886c3587b3eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:13:53.324769 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.324739 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f8242e6-6f42-4da2-a95e-886c3587b3eb-kube-api-access-vdfq2" (OuterVolumeSpecName: "kube-api-access-vdfq2") pod "6f8242e6-6f42-4da2-a95e-886c3587b3eb" (UID: "6f8242e6-6f42-4da2-a95e-886c3587b3eb"). InnerVolumeSpecName "kube-api-access-vdfq2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:13:53.424198 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.424154 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f8242e6-6f42-4da2-a95e-886c3587b3eb-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:13:53.424198 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.424187 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdfq2\" (UniqueName: \"kubernetes.io/projected/6f8242e6-6f42-4da2-a95e-886c3587b3eb-kube-api-access-vdfq2\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:13:53.471877 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.471767 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:53.573589 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.573539 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:53.664885 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.664848 2584 generic.go:356] "Generic (PLEG): container finished" podID="6f8242e6-6f42-4da2-a95e-886c3587b3eb" containerID="8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab" exitCode=0 Sep 19 06:13:53.665089 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.664897 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" Sep 19 06:13:53.665089 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.664929 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" event={"ID":"6f8242e6-6f42-4da2-a95e-886c3587b3eb","Type":"ContainerDied","Data":"8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab"} Sep 19 06:13:53.665089 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.664983 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74" event={"ID":"6f8242e6-6f42-4da2-a95e-886c3587b3eb","Type":"ContainerDied","Data":"67182c34b844bba885cd1e8ce4b9e13d213cfca4874b85f388263b9ab82ac3e7"} Sep 19 06:13:53.665089 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.665007 2584 scope.go:117] "RemoveContainer" containerID="8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab" Sep 19 06:13:53.674368 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.674348 2584 scope.go:117] "RemoveContainer" containerID="8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab" Sep 19 06:13:53.674698 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:13:53.674676 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab\": container with ID starting with 8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab not found: ID does not exist" containerID="8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab" Sep 19 06:13:53.674797 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.674709 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab"} err="failed to get container status \"8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab\": rpc error: code = NotFound desc = could not find container \"8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab\": container with ID starting with 8a2244f08efe0a9c8a13f6afec47260e2d16fa9ce2171e41d976e4d16503e3ab not found: ID does not exist" Sep 19 06:13:53.686148 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.686115 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74"] Sep 19 06:13:53.689413 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:53.689376 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7142e3-79cb9d7b6-k7j74"] Sep 19 06:13:54.181257 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:54.181232 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:54.267778 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:54.267746 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f8242e6-6f42-4da2-a95e-886c3587b3eb" path="/var/lib/kubelet/pods/6f8242e6-6f42-4da2-a95e-886c3587b3eb/volumes" Sep 19 06:13:54.268119 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:54.268106 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba0412ac-8296-4e11-a3aa-65baf26ab2df" path="/var/lib/kubelet/pods/ba0412ac-8296-4e11-a3aa-65baf26ab2df/volumes" Sep 19 06:13:54.855447 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:54.855415 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:55.697744 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:55.697710 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:13:58.548028 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:13:58.547990 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:12.677876 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:12.677832 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:14.845072 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:14.845041 2584 scope.go:117] "RemoveContainer" containerID="85db361ea6d95de1218a5b763a91cb08228c685e7f274ea2b04d75013bb7efd1" Sep 19 06:14:34.684422 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:34.684365 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:35.851874 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:35.851843 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:41.294342 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.294303 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:41.356394 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.356345 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:41.566536 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.566437 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n"] Sep 19 06:14:41.567104 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.567049 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" containerID="cri-o://e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37" gracePeriod=30 Sep 19 06:14:41.576340 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.576312 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:41.952073 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.952036 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv"] Sep 19 06:14:41.952514 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.952500 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba0412ac-8296-4e11-a3aa-65baf26ab2df" containerName="revoke-keys" Sep 19 06:14:41.952514 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.952515 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba0412ac-8296-4e11-a3aa-65baf26ab2df" containerName="revoke-keys" Sep 19 06:14:41.952633 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.952531 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f8242e6-6f42-4da2-a95e-886c3587b3eb" containerName="payload-pre-processing" Sep 19 06:14:41.952633 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.952537 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f8242e6-6f42-4da2-a95e-886c3587b3eb" containerName="payload-pre-processing" Sep 19 06:14:41.952633 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.952618 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f8242e6-6f42-4da2-a95e-886c3587b3eb" containerName="payload-pre-processing" Sep 19 06:14:41.952633 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.952630 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba0412ac-8296-4e11-a3aa-65baf26ab2df" containerName="revoke-keys" Sep 19 06:14:41.955815 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.955800 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv" Sep 19 06:14:41.957992 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.957970 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:14:41.961994 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:41.961944 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv"] Sep 19 06:14:42.023880 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:42.023843 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f55d516d-21ec-45de-8465-c2bc4481cedb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv\" (UID: \"f55d516d-21ec-45de-8465-c2bc4481cedb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv" Sep 19 06:14:42.107551 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:42.107499 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:42.124733 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:42.124695 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f55d516d-21ec-45de-8465-c2bc4481cedb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv\" (UID: \"f55d516d-21ec-45de-8465-c2bc4481cedb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv" Sep 19 06:14:42.125393 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:42.125358 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f55d516d-21ec-45de-8465-c2bc4481cedb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv\" (UID: \"f55d516d-21ec-45de-8465-c2bc4481cedb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv" Sep 19 06:14:42.270144 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:42.270066 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv" Sep 19 06:14:42.545989 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:42.545965 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv"] Sep 19 06:14:42.548724 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:14:42.548694 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf55d516d_21ec_45de_8465_c2bc4481cedb.slice/crio-460d2e0f45cc5c2af0d3a24ceb77f2526feec45ea29ea8632ba6a414c3df62b1 WatchSource:0}: Error finding container 460d2e0f45cc5c2af0d3a24ceb77f2526feec45ea29ea8632ba6a414c3df62b1: Status 404 returned error can't find the container with id 460d2e0f45cc5c2af0d3a24ceb77f2526feec45ea29ea8632ba6a414c3df62b1 Sep 19 06:14:42.892310 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:42.892280 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:42.938972 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:42.938935 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv" event={"ID":"f55d516d-21ec-45de-8465-c2bc4481cedb","Type":"ContainerStarted","Data":"c8fec2aef3159cc21a33cc99994ec436f2afc7ecd6de284ea57ee0ef2d6d0a83"} Sep 19 06:14:42.939143 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:42.938987 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv" event={"ID":"f55d516d-21ec-45de-8465-c2bc4481cedb","Type":"ContainerStarted","Data":"460d2e0f45cc5c2af0d3a24ceb77f2526feec45ea29ea8632ba6a414c3df62b1"} Sep 19 06:14:42.955770 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:42.955713 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv" podStartSLOduration=1.9556957339999999 podStartE2EDuration="1.955695734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:14:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:14:42.953256927 +0000 UTC m=+2010.946484460" watchObservedRunningTime="2026-09-19 06:14:42.955695734 +0000 UTC m=+2010.948923264" Sep 19 06:14:43.699274 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:43.699239 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:43.946390 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:43.946349 2584 generic.go:356] "Generic (PLEG): container finished" podID="f55d516d-21ec-45de-8465-c2bc4481cedb" containerID="c8fec2aef3159cc21a33cc99994ec436f2afc7ecd6de284ea57ee0ef2d6d0a83" exitCode=0 Sep 19 06:14:43.946629 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:43.946454 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv" event={"ID":"f55d516d-21ec-45de-8465-c2bc4481cedb","Type":"ContainerDied","Data":"c8fec2aef3159cc21a33cc99994ec436f2afc7ecd6de284ea57ee0ef2d6d0a83"} Sep 19 06:14:44.336789 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:44.336754 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:44.985441 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:44.985389 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:45.093382 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:45.093351 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv" Sep 19 06:14:45.154568 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:45.154529 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f55d516d-21ec-45de-8465-c2bc4481cedb-maas-api-service-ca\") pod \"f55d516d-21ec-45de-8465-c2bc4481cedb\" (UID: \"f55d516d-21ec-45de-8465-c2bc4481cedb\") " Sep 19 06:14:45.154952 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:45.154925 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f55d516d-21ec-45de-8465-c2bc4481cedb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f55d516d-21ec-45de-8465-c2bc4481cedb" (UID: "f55d516d-21ec-45de-8465-c2bc4481cedb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:14:45.256511 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:45.256385 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f55d516d-21ec-45de-8465-c2bc4481cedb-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:45.958740 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:45.958712 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv" Sep 19 06:14:45.958922 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:45.958759 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv" event={"ID":"f55d516d-21ec-45de-8465-c2bc4481cedb","Type":"ContainerDied","Data":"460d2e0f45cc5c2af0d3a24ceb77f2526feec45ea29ea8632ba6a414c3df62b1"} Sep 19 06:14:45.958922 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:45.958806 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="460d2e0f45cc5c2af0d3a24ceb77f2526feec45ea29ea8632ba6a414c3df62b1" Sep 19 06:14:46.825546 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.825517 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:14:46.966499 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.966455 2584 generic.go:356] "Generic (PLEG): container finished" podID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerID="e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37" exitCode=0 Sep 19 06:14:46.966672 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.966510 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" Sep 19 06:14:46.966672 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.966527 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" event={"ID":"9059cab0-dc5c-4595-a718-18e2ffd10bb1","Type":"ContainerDied","Data":"e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37"} Sep 19 06:14:46.966672 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.966614 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n" event={"ID":"9059cab0-dc5c-4595-a718-18e2ffd10bb1","Type":"ContainerDied","Data":"f2525c447c2b1a96b86002da4132dd0c18b57ab5fc83393acb5b2d04e96dad5b"} Sep 19 06:14:46.966672 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.966641 2584 scope.go:117] "RemoveContainer" containerID="e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37" Sep 19 06:14:46.971781 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.971756 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-workload-certs\") pod \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " Sep 19 06:14:46.971892 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.971798 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snmz9\" (UniqueName: \"kubernetes.io/projected/9059cab0-dc5c-4595-a718-18e2ffd10bb1-kube-api-access-snmz9\") pod \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " Sep 19 06:14:46.971892 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.971840 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-envoy\") pod \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " Sep 19 06:14:46.971892 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.971879 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-workload-socket\") pod \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " Sep 19 06:14:46.972053 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.971941 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-token\") pod \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " Sep 19 06:14:46.972111 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.972073 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9059cab0-dc5c-4595-a718-18e2ffd10bb1" (UID: "9059cab0-dc5c-4595-a718-18e2ffd10bb1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:14:46.972259 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.972234 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9059cab0-dc5c-4595-a718-18e2ffd10bb1" (UID: "9059cab0-dc5c-4595-a718-18e2ffd10bb1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:14:46.972482 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.972340 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-data\") pod \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " Sep 19 06:14:46.972482 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.972451 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-credential-socket\") pod \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " Sep 19 06:14:46.972654 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.972493 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istiod-ca-cert\") pod \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " Sep 19 06:14:46.972654 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.972584 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-podinfo\") pod \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\" (UID: \"9059cab0-dc5c-4595-a718-18e2ffd10bb1\") " Sep 19 06:14:46.972834 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.972762 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-data" (OuterVolumeSpecName: "istio-data") pod "9059cab0-dc5c-4595-a718-18e2ffd10bb1" (UID: "9059cab0-dc5c-4595-a718-18e2ffd10bb1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:14:46.973351 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.973327 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9059cab0-dc5c-4595-a718-18e2ffd10bb1" (UID: "9059cab0-dc5c-4595-a718-18e2ffd10bb1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:14:46.973732 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.973714 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9059cab0-dc5c-4595-a718-18e2ffd10bb1" (UID: "9059cab0-dc5c-4595-a718-18e2ffd10bb1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:14:46.974174 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.974155 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-workload-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:46.974271 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.974260 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-data\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:46.974352 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.974341 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-credential-socket\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:46.974461 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.974450 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istiod-ca-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:46.974544 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.974534 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-workload-certs\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:46.974624 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.974572 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-token" (OuterVolumeSpecName: "istio-token") pod "9059cab0-dc5c-4595-a718-18e2ffd10bb1" (UID: "9059cab0-dc5c-4595-a718-18e2ffd10bb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:14:46.978622 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.978592 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9059cab0-dc5c-4595-a718-18e2ffd10bb1" (UID: "9059cab0-dc5c-4595-a718-18e2ffd10bb1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 06:14:46.978723 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.978620 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9059cab0-dc5c-4595-a718-18e2ffd10bb1" (UID: "9059cab0-dc5c-4595-a718-18e2ffd10bb1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 06:14:46.978723 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.978627 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9059cab0-dc5c-4595-a718-18e2ffd10bb1-kube-api-access-snmz9" (OuterVolumeSpecName: "kube-api-access-snmz9") pod "9059cab0-dc5c-4595-a718-18e2ffd10bb1" (UID: "9059cab0-dc5c-4595-a718-18e2ffd10bb1"). InnerVolumeSpecName "kube-api-access-snmz9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:14:46.980808 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.980783 2584 scope.go:117] "RemoveContainer" containerID="219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e" Sep 19 06:14:46.997009 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.996988 2584 scope.go:117] "RemoveContainer" containerID="e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37" Sep 19 06:14:46.997324 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:14:46.997297 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37\": container with ID starting with e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37 not found: ID does not exist" containerID="e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37" Sep 19 06:14:46.997435 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.997327 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37"} err="failed to get container status \"e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37\": rpc error: code = NotFound desc = could not find container \"e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37\": container with ID starting with e420673e34e46a14ea80a8d5492365c9b5b062df529a9f7daa47163bc30b6a37 not found: ID does not exist" Sep 19 06:14:46.997435 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.997346 2584 scope.go:117] "RemoveContainer" containerID="219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e" Sep 19 06:14:46.997645 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:14:46.997626 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e\": container with ID starting with 219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e not found: ID does not exist" containerID="219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e" Sep 19 06:14:46.997689 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:46.997649 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e"} err="failed to get container status \"219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e\": rpc error: code = NotFound desc = could not find container \"219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e\": container with ID starting with 219ffb5ab6e94530d052923c03d97cdebfaa2265148973f2682e98b605dd518e not found: ID does not exist" Sep 19 06:14:47.075852 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:47.075818 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-envoy\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:47.075852 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:47.075845 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-token\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:47.075852 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:47.075853 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9059cab0-dc5c-4595-a718-18e2ffd10bb1-istio-podinfo\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:47.076093 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:47.075863 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snmz9\" (UniqueName: \"kubernetes.io/projected/9059cab0-dc5c-4595-a718-18e2ffd10bb1-kube-api-access-snmz9\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:47.289093 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:47.289056 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n"] Sep 19 06:14:47.292995 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:47.292961 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-7142e3-openshift-default-84f6d7f95d-ht84n"] Sep 19 06:14:48.267831 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:48.267795 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" path="/var/lib/kubelet/pods/9059cab0-dc5c-4595-a718-18e2ffd10bb1/volumes" Sep 19 06:14:52.524440 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:52.524377 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:52.573227 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:52.573195 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp"] Sep 19 06:14:52.573439 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:52.573412 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" podUID="525145f8-7541-48c2-82b8-0b168e17cc25" containerName="payload-pre-processing" containerID="cri-o://339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e" gracePeriod=30 Sep 19 06:14:52.577345 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:52.577318 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh"] Sep 19 06:14:52.577590 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:52.577563 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" podUID="b8f87f58-b4da-4270-90fb-219b04573ed7" containerName="payload-processing" containerID="cri-o://63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137" gracePeriod=30 Sep 19 06:14:52.660013 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:52.659940 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:52.766672 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:52.766636 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:52.815804 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:52.815764 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv"] Sep 19 06:14:52.819180 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:52.819149 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ebb0f4-96jsv"] Sep 19 06:14:52.833276 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:52.833251 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:52.869340 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:52.869315 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" Sep 19 06:14:52.872490 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:52.872468 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" Sep 19 06:14:53.002817 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.002780 2584 generic.go:356] "Generic (PLEG): container finished" podID="525145f8-7541-48c2-82b8-0b168e17cc25" containerID="339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e" exitCode=0 Sep 19 06:14:53.003018 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.002828 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" Sep 19 06:14:53.003018 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.002868 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" event={"ID":"525145f8-7541-48c2-82b8-0b168e17cc25","Type":"ContainerDied","Data":"339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e"} Sep 19 06:14:53.003018 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.002926 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp" event={"ID":"525145f8-7541-48c2-82b8-0b168e17cc25","Type":"ContainerDied","Data":"47724e3c8deb6f462bed403027d159e2d7fa8dc42c6f76f51df1faab2b06a1c1"} Sep 19 06:14:53.003018 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.002954 2584 scope.go:117] "RemoveContainer" containerID="339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e" Sep 19 06:14:53.004264 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.004246 2584 generic.go:356] "Generic (PLEG): container finished" podID="b8f87f58-b4da-4270-90fb-219b04573ed7" containerID="63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137" exitCode=0 Sep 19 06:14:53.004334 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.004292 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" Sep 19 06:14:53.004373 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.004335 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" event={"ID":"b8f87f58-b4da-4270-90fb-219b04573ed7","Type":"ContainerDied","Data":"63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137"} Sep 19 06:14:53.004441 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.004386 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh" event={"ID":"b8f87f58-b4da-4270-90fb-219b04573ed7","Type":"ContainerDied","Data":"2658d2b848e5f604d25988841d54295d3fd4b3cc90c3daf0c216bdd076a0cc7f"} Sep 19 06:14:53.012609 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.012586 2584 scope.go:117] "RemoveContainer" containerID="339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e" Sep 19 06:14:53.012916 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:14:53.012898 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e\": container with ID starting with 339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e not found: ID does not exist" containerID="339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e" Sep 19 06:14:53.013029 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.012923 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e"} err="failed to get container status \"339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e\": rpc error: code = NotFound desc = could not find container \"339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e\": container with ID starting with 339268567f529922dd2b65882de8df746dbe5ab8f4958ccda27c285158ef272e not found: ID does not exist" Sep 19 06:14:53.013029 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.012938 2584 scope.go:117] "RemoveContainer" containerID="63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137" Sep 19 06:14:53.021465 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.021443 2584 scope.go:117] "RemoveContainer" containerID="63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137" Sep 19 06:14:53.021791 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:14:53.021766 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137\": container with ID starting with 63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137 not found: ID does not exist" containerID="63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137" Sep 19 06:14:53.021859 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.021800 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137"} err="failed to get container status \"63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137\": rpc error: code = NotFound desc = could not find container \"63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137\": container with ID starting with 63cdc633d376e666ac89895d89bbfdff68cde202f344f1e0e60984a5af5d2137 not found: ID does not exist" Sep 19 06:14:53.038784 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.038717 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8f87f58-b4da-4270-90fb-219b04573ed7-plugins-config-volume\") pod \"b8f87f58-b4da-4270-90fb-219b04573ed7\" (UID: \"b8f87f58-b4da-4270-90fb-219b04573ed7\") " Sep 19 06:14:53.038784 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.038778 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/525145f8-7541-48c2-82b8-0b168e17cc25-plugins-config-volume\") pod \"525145f8-7541-48c2-82b8-0b168e17cc25\" (UID: \"525145f8-7541-48c2-82b8-0b168e17cc25\") " Sep 19 06:14:53.038982 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.038821 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxsp5\" (UniqueName: \"kubernetes.io/projected/b8f87f58-b4da-4270-90fb-219b04573ed7-kube-api-access-fxsp5\") pod \"b8f87f58-b4da-4270-90fb-219b04573ed7\" (UID: \"b8f87f58-b4da-4270-90fb-219b04573ed7\") " Sep 19 06:14:53.038982 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.038846 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwqqk\" (UniqueName: \"kubernetes.io/projected/525145f8-7541-48c2-82b8-0b168e17cc25-kube-api-access-wwqqk\") pod \"525145f8-7541-48c2-82b8-0b168e17cc25\" (UID: \"525145f8-7541-48c2-82b8-0b168e17cc25\") " Sep 19 06:14:53.039180 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.039151 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8f87f58-b4da-4270-90fb-219b04573ed7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8f87f58-b4da-4270-90fb-219b04573ed7" (UID: "b8f87f58-b4da-4270-90fb-219b04573ed7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:14:53.039281 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.039203 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/525145f8-7541-48c2-82b8-0b168e17cc25-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "525145f8-7541-48c2-82b8-0b168e17cc25" (UID: "525145f8-7541-48c2-82b8-0b168e17cc25"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:14:53.040953 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.040934 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8f87f58-b4da-4270-90fb-219b04573ed7-kube-api-access-fxsp5" (OuterVolumeSpecName: "kube-api-access-fxsp5") pod "b8f87f58-b4da-4270-90fb-219b04573ed7" (UID: "b8f87f58-b4da-4270-90fb-219b04573ed7"). InnerVolumeSpecName "kube-api-access-fxsp5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:14:53.041024 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.041004 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/525145f8-7541-48c2-82b8-0b168e17cc25-kube-api-access-wwqqk" (OuterVolumeSpecName: "kube-api-access-wwqqk") pod "525145f8-7541-48c2-82b8-0b168e17cc25" (UID: "525145f8-7541-48c2-82b8-0b168e17cc25"). InnerVolumeSpecName "kube-api-access-wwqqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:14:53.140119 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.140077 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8f87f58-b4da-4270-90fb-219b04573ed7-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:53.140119 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.140112 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/525145f8-7541-48c2-82b8-0b168e17cc25-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:53.140119 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.140121 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxsp5\" (UniqueName: \"kubernetes.io/projected/b8f87f58-b4da-4270-90fb-219b04573ed7-kube-api-access-fxsp5\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:53.140119 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.140130 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwqqk\" (UniqueName: \"kubernetes.io/projected/525145f8-7541-48c2-82b8-0b168e17cc25-kube-api-access-wwqqk\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:14:53.192462 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.192388 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:53.292830 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.292743 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:53.328065 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.328036 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh"] Sep 19 06:14:53.331830 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.331803 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-ebb0f4-5cc8db988f-bgchh"] Sep 19 06:14:53.342001 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.341975 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp"] Sep 19 06:14:53.345434 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.345409 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ebb0f4-56554d89d6-wj9wp"] Sep 19 06:14:53.992794 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:53.992706 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:54.267782 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:54.267698 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="525145f8-7541-48c2-82b8-0b168e17cc25" path="/var/lib/kubelet/pods/525145f8-7541-48c2-82b8-0b168e17cc25/volumes" Sep 19 06:14:54.268086 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:54.268071 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8f87f58-b4da-4270-90fb-219b04573ed7" path="/var/lib/kubelet/pods/b8f87f58-b4da-4270-90fb-219b04573ed7/volumes" Sep 19 06:14:54.268366 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:54.268355 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f55d516d-21ec-45de-8465-c2bc4481cedb" path="/var/lib/kubelet/pods/f55d516d-21ec-45de-8465-c2bc4481cedb/volumes" Sep 19 06:14:54.609427 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:54.609353 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:58.726231 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:58.726196 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:58.826483 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:58.826447 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:59.004006 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.003918 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:59.339915 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.339872 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc"] Sep 19 06:14:59.340363 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340345 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="525145f8-7541-48c2-82b8-0b168e17cc25" containerName="payload-pre-processing" Sep 19 06:14:59.340363 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340362 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="525145f8-7541-48c2-82b8-0b168e17cc25" containerName="payload-pre-processing" Sep 19 06:14:59.340619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340384 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8f87f58-b4da-4270-90fb-219b04573ed7" containerName="payload-processing" Sep 19 06:14:59.340619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340393 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8f87f58-b4da-4270-90fb-219b04573ed7" containerName="payload-processing" Sep 19 06:14:59.340619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340414 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f55d516d-21ec-45de-8465-c2bc4481cedb" containerName="revoke-keys" Sep 19 06:14:59.340619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340420 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f55d516d-21ec-45de-8465-c2bc4481cedb" containerName="revoke-keys" Sep 19 06:14:59.340619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340431 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" Sep 19 06:14:59.340619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340437 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" Sep 19 06:14:59.340619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340456 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" Sep 19 06:14:59.340619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340465 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" Sep 19 06:14:59.340619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340563 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" Sep 19 06:14:59.340619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340580 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="525145f8-7541-48c2-82b8-0b168e17cc25" containerName="payload-pre-processing" Sep 19 06:14:59.340619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340594 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8f87f58-b4da-4270-90fb-219b04573ed7" containerName="payload-processing" Sep 19 06:14:59.340619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.340603 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f55d516d-21ec-45de-8465-c2bc4481cedb" containerName="revoke-keys" Sep 19 06:14:59.345189 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.345165 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc" Sep 19 06:14:59.347959 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.347931 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:14:59.349432 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.349382 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc"] Sep 19 06:14:59.403019 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.402982 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2867a4f7-fe22-45ac-9281-8610ab0e92b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc\" (UID: \"2867a4f7-fe22-45ac-9281-8610ab0e92b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc" Sep 19 06:14:59.504489 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.504447 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2867a4f7-fe22-45ac-9281-8610ab0e92b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc\" (UID: \"2867a4f7-fe22-45ac-9281-8610ab0e92b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc" Sep 19 06:14:59.505201 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.505174 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2867a4f7-fe22-45ac-9281-8610ab0e92b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc\" (UID: \"2867a4f7-fe22-45ac-9281-8610ab0e92b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc" Sep 19 06:14:59.507461 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.507434 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:14:59.657324 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.657237 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc" Sep 19 06:14:59.931315 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:14:59.931284 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc"] Sep 19 06:14:59.933425 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:14:59.933370 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2867a4f7_fe22_45ac_9281_8610ab0e92b3.slice/crio-ce2d2daf530092c0b9404f7edd1bbe1a0d9a95f348e74d6159727865ce8284d2 WatchSource:0}: Error finding container ce2d2daf530092c0b9404f7edd1bbe1a0d9a95f348e74d6159727865ce8284d2: Status 404 returned error can't find the container with id ce2d2daf530092c0b9404f7edd1bbe1a0d9a95f348e74d6159727865ce8284d2 Sep 19 06:15:00.045059 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:00.045021 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc" event={"ID":"2867a4f7-fe22-45ac-9281-8610ab0e92b3","Type":"ContainerStarted","Data":"ce2d2daf530092c0b9404f7edd1bbe1a0d9a95f348e74d6159727865ce8284d2"} Sep 19 06:15:00.141337 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:00.141305 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj"] Sep 19 06:15:00.141842 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:00.141828 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9059cab0-dc5c-4595-a718-18e2ffd10bb1" containerName="istio-proxy" Sep 19 06:15:00.144958 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:00.144941 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj" Sep 19 06:15:00.153643 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:00.153552 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj"] Sep 19 06:15:00.159342 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:00.159312 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj" Sep 19 06:15:00.314544 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:00.314507 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:00.436484 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:00.436451 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj"] Sep 19 06:15:00.437569 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:15:00.437542 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70740029_13a4_48bb_aa2a_624c3b829963.slice/crio-f6af0f3d62abeb72a43cc6c9404f132704c6dead56decd026e251ba82afa30e2 WatchSource:0}: Error finding container f6af0f3d62abeb72a43cc6c9404f132704c6dead56decd026e251ba82afa30e2: Status 404 returned error can't find the container with id f6af0f3d62abeb72a43cc6c9404f132704c6dead56decd026e251ba82afa30e2 Sep 19 06:15:01.007961 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:01.007860 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:01.052619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:01.052580 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj" event={"ID":"70740029-13a4-48bb-aa2a-624c3b829963","Type":"ContainerStarted","Data":"8d5b5a32e667aaf8aa3089bbf78f857d4a7a5c6ada0caa852f9501cdcaa0d5a0"} Sep 19 06:15:01.052808 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:01.052638 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj" event={"ID":"70740029-13a4-48bb-aa2a-624c3b829963","Type":"ContainerStarted","Data":"f6af0f3d62abeb72a43cc6c9404f132704c6dead56decd026e251ba82afa30e2"} Sep 19 06:15:01.054202 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:01.054160 2584 generic.go:356] "Generic (PLEG): container finished" podID="2867a4f7-fe22-45ac-9281-8610ab0e92b3" containerID="4d31474cfef0e7562a7a1a2c37b720628a9aff2507868e1b8afbd78e3fb4f0b3" exitCode=0 Sep 19 06:15:01.054335 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:01.054240 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc" event={"ID":"2867a4f7-fe22-45ac-9281-8610ab0e92b3","Type":"ContainerDied","Data":"4d31474cfef0e7562a7a1a2c37b720628a9aff2507868e1b8afbd78e3fb4f0b3"} Sep 19 06:15:01.068597 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:01.068542 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj" podStartSLOduration=1.0685261129999999 podStartE2EDuration="1.068526113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:15:01.066789934 +0000 UTC m=+2029.060017466" watchObservedRunningTime="2026-09-19 06:15:01.068526113 +0000 UTC m=+2029.061753701" Sep 19 06:15:01.808159 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:01.808118 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:02.061689 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:02.061596 2584 generic.go:356] "Generic (PLEG): container finished" podID="70740029-13a4-48bb-aa2a-624c3b829963" containerID="8d5b5a32e667aaf8aa3089bbf78f857d4a7a5c6ada0caa852f9501cdcaa0d5a0" exitCode=0 Sep 19 06:15:02.061689 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:02.061659 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj" event={"ID":"70740029-13a4-48bb-aa2a-624c3b829963","Type":"ContainerDied","Data":"8d5b5a32e667aaf8aa3089bbf78f857d4a7a5c6ada0caa852f9501cdcaa0d5a0"} Sep 19 06:15:02.219366 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:02.219339 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc" Sep 19 06:15:02.229839 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:02.229814 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2867a4f7-fe22-45ac-9281-8610ab0e92b3-maas-api-service-ca\") pod \"2867a4f7-fe22-45ac-9281-8610ab0e92b3\" (UID: \"2867a4f7-fe22-45ac-9281-8610ab0e92b3\") " Sep 19 06:15:02.230151 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:02.230125 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2867a4f7-fe22-45ac-9281-8610ab0e92b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2867a4f7-fe22-45ac-9281-8610ab0e92b3" (UID: "2867a4f7-fe22-45ac-9281-8610ab0e92b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:15:02.331585 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:02.331540 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2867a4f7-fe22-45ac-9281-8610ab0e92b3-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:15:03.068285 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:03.068253 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc" Sep 19 06:15:03.068709 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:03.068280 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc" event={"ID":"2867a4f7-fe22-45ac-9281-8610ab0e92b3","Type":"ContainerDied","Data":"ce2d2daf530092c0b9404f7edd1bbe1a0d9a95f348e74d6159727865ce8284d2"} Sep 19 06:15:03.068709 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:03.068329 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce2d2daf530092c0b9404f7edd1bbe1a0d9a95f348e74d6159727865ce8284d2" Sep 19 06:15:03.202079 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:03.202051 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj" Sep 19 06:15:04.076785 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:04.076755 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj" Sep 19 06:15:04.077176 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:04.076786 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj" event={"ID":"70740029-13a4-48bb-aa2a-624c3b829963","Type":"ContainerDied","Data":"f6af0f3d62abeb72a43cc6c9404f132704c6dead56decd026e251ba82afa30e2"} Sep 19 06:15:04.077176 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:04.076836 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6af0f3d62abeb72a43cc6c9404f132704c6dead56decd026e251ba82afa30e2" Sep 19 06:15:09.777840 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:09.777804 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj"] Sep 19 06:15:09.779161 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:09.779135 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:09.780296 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:09.780269 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-888f75-29829975-psklj"] Sep 19 06:15:09.797680 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:09.797642 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt"] Sep 19 06:15:09.797908 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:09.797875 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" podUID="75769fa3-8ead-4660-b9ee-3e5d68176c7e" containerName="payload-processing" containerID="cri-o://edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4" gracePeriod=30 Sep 19 06:15:09.840797 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:09.840766 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:09.940699 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:09.940666 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:09.951378 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:09.951335 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc"] Sep 19 06:15:09.953287 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:09.953229 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-888f75-qn4pc"] Sep 19 06:15:09.997704 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:09.997671 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:10.067060 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.067033 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" Sep 19 06:15:10.109697 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.109667 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75769fa3-8ead-4660-b9ee-3e5d68176c7e-plugins-config-volume\") pod \"75769fa3-8ead-4660-b9ee-3e5d68176c7e\" (UID: \"75769fa3-8ead-4660-b9ee-3e5d68176c7e\") " Sep 19 06:15:10.109897 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.109745 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prtdc\" (UniqueName: \"kubernetes.io/projected/75769fa3-8ead-4660-b9ee-3e5d68176c7e-kube-api-access-prtdc\") pod \"75769fa3-8ead-4660-b9ee-3e5d68176c7e\" (UID: \"75769fa3-8ead-4660-b9ee-3e5d68176c7e\") " Sep 19 06:15:10.110155 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.110126 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75769fa3-8ead-4660-b9ee-3e5d68176c7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "75769fa3-8ead-4660-b9ee-3e5d68176c7e" (UID: "75769fa3-8ead-4660-b9ee-3e5d68176c7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:15:10.112199 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.112165 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75769fa3-8ead-4660-b9ee-3e5d68176c7e-kube-api-access-prtdc" (OuterVolumeSpecName: "kube-api-access-prtdc") pod "75769fa3-8ead-4660-b9ee-3e5d68176c7e" (UID: "75769fa3-8ead-4660-b9ee-3e5d68176c7e"). InnerVolumeSpecName "kube-api-access-prtdc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:15:10.112817 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.112790 2584 generic.go:356] "Generic (PLEG): container finished" podID="75769fa3-8ead-4660-b9ee-3e5d68176c7e" containerID="edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4" exitCode=0 Sep 19 06:15:10.112946 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.112848 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" event={"ID":"75769fa3-8ead-4660-b9ee-3e5d68176c7e","Type":"ContainerDied","Data":"edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4"} Sep 19 06:15:10.112946 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.112897 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" event={"ID":"75769fa3-8ead-4660-b9ee-3e5d68176c7e","Type":"ContainerDied","Data":"c9714725b202164b09e8f6c3660798446f5fcaf2cc3756a1edec526239d6f488"} Sep 19 06:15:10.112946 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.112926 2584 scope.go:117] "RemoveContainer" containerID="edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4" Sep 19 06:15:10.113101 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.113088 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt" Sep 19 06:15:10.126529 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.126504 2584 scope.go:117] "RemoveContainer" containerID="edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4" Sep 19 06:15:10.126883 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:15:10.126862 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4\": container with ID starting with edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4 not found: ID does not exist" containerID="edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4" Sep 19 06:15:10.126975 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.126891 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4"} err="failed to get container status \"edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4\": rpc error: code = NotFound desc = could not find container \"edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4\": container with ID starting with edf0891a319dceb6ac3ebcc1d4b2811fab76c63a6628eb976465fba922f7c7b4 not found: ID does not exist" Sep 19 06:15:10.137289 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.137254 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt"] Sep 19 06:15:10.141013 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.140986 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-888f75-7678cffd5f-sgnqt"] Sep 19 06:15:10.211525 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.211489 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prtdc\" (UniqueName: \"kubernetes.io/projected/75769fa3-8ead-4660-b9ee-3e5d68176c7e-kube-api-access-prtdc\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:15:10.211525 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.211521 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75769fa3-8ead-4660-b9ee-3e5d68176c7e-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:15:10.267662 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.267629 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2867a4f7-fe22-45ac-9281-8610ab0e92b3" path="/var/lib/kubelet/pods/2867a4f7-fe22-45ac-9281-8610ab0e92b3/volumes" Sep 19 06:15:10.267949 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.267937 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70740029-13a4-48bb-aa2a-624c3b829963" path="/var/lib/kubelet/pods/70740029-13a4-48bb-aa2a-624c3b829963/volumes" Sep 19 06:15:10.268228 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.268216 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75769fa3-8ead-4660-b9ee-3e5d68176c7e" path="/var/lib/kubelet/pods/75769fa3-8ead-4660-b9ee-3e5d68176c7e/volumes" Sep 19 06:15:10.342949 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.342918 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:10.442881 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:10.442845 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:11.089071 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:11.089038 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:16.271377 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.271344 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:16.331611 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.331576 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:16.548386 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.548212 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:16.885083 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.885044 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8"] Sep 19 06:15:16.885518 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.885502 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70740029-13a4-48bb-aa2a-624c3b829963" containerName="cleanup" Sep 19 06:15:16.885518 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.885517 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="70740029-13a4-48bb-aa2a-624c3b829963" containerName="cleanup" Sep 19 06:15:16.885674 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.885543 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75769fa3-8ead-4660-b9ee-3e5d68176c7e" containerName="payload-processing" Sep 19 06:15:16.885674 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.885549 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="75769fa3-8ead-4660-b9ee-3e5d68176c7e" containerName="payload-processing" Sep 19 06:15:16.885674 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.885558 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2867a4f7-fe22-45ac-9281-8610ab0e92b3" containerName="revoke-keys" Sep 19 06:15:16.885674 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.885564 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2867a4f7-fe22-45ac-9281-8610ab0e92b3" containerName="revoke-keys" Sep 19 06:15:16.885674 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.885632 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2867a4f7-fe22-45ac-9281-8610ab0e92b3" containerName="revoke-keys" Sep 19 06:15:16.885674 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.885641 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="70740029-13a4-48bb-aa2a-624c3b829963" containerName="cleanup" Sep 19 06:15:16.885674 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.885650 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="75769fa3-8ead-4660-b9ee-3e5d68176c7e" containerName="payload-processing" Sep 19 06:15:16.890345 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.890324 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8" Sep 19 06:15:16.892896 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.892847 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-knzkr\"" Sep 19 06:15:16.895195 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.895170 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8"] Sep 19 06:15:16.975247 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:16.975212 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6564aa72-12bb-41e8-9e12-0a88dc25b04d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8\" (UID: \"6564aa72-12bb-41e8-9e12-0a88dc25b04d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8" Sep 19 06:15:17.076596 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:17.076554 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6564aa72-12bb-41e8-9e12-0a88dc25b04d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8\" (UID: \"6564aa72-12bb-41e8-9e12-0a88dc25b04d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8" Sep 19 06:15:17.077214 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:17.077191 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6564aa72-12bb-41e8-9e12-0a88dc25b04d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8\" (UID: \"6564aa72-12bb-41e8-9e12-0a88dc25b04d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8" Sep 19 06:15:17.204314 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:17.204221 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8" Sep 19 06:15:17.483234 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:17.483203 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8"] Sep 19 06:15:17.485732 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:15:17.485705 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6564aa72_12bb_41e8_9e12_0a88dc25b04d.slice/crio-f2def852ae6f56efe98183233ee5d44850dc39e01c6b3155424361cacb2208c0 WatchSource:0}: Error finding container f2def852ae6f56efe98183233ee5d44850dc39e01c6b3155424361cacb2208c0: Status 404 returned error can't find the container with id f2def852ae6f56efe98183233ee5d44850dc39e01c6b3155424361cacb2208c0 Sep 19 06:15:17.697678 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:17.697647 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:18.159080 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:18.159042 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8" event={"ID":"6564aa72-12bb-41e8-9e12-0a88dc25b04d","Type":"ContainerStarted","Data":"2e7c9e3a8bbd3fc32949f9bd39698b033e383bfe96fe300d4aaba1c02b65b71b"} Sep 19 06:15:18.159259 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:18.159097 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8" event={"ID":"6564aa72-12bb-41e8-9e12-0a88dc25b04d","Type":"ContainerStarted","Data":"f2def852ae6f56efe98183233ee5d44850dc39e01c6b3155424361cacb2208c0"} Sep 19 06:15:18.175602 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:18.175551 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8" podStartSLOduration=2.175533159 podStartE2EDuration="2.175533159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:15:18.173421321 +0000 UTC m=+2046.166648854" watchObservedRunningTime="2026-09-19 06:15:18.175533159 +0000 UTC m=+2046.168760690" Sep 19 06:15:18.320529 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:18.320501 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:19.166450 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:19.166413 2584 generic.go:356] "Generic (PLEG): container finished" podID="6564aa72-12bb-41e8-9e12-0a88dc25b04d" containerID="2e7c9e3a8bbd3fc32949f9bd39698b033e383bfe96fe300d4aaba1c02b65b71b" exitCode=0 Sep 19 06:15:19.166870 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:19.166497 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8" event={"ID":"6564aa72-12bb-41e8-9e12-0a88dc25b04d","Type":"ContainerDied","Data":"2e7c9e3a8bbd3fc32949f9bd39698b033e383bfe96fe300d4aaba1c02b65b71b"} Sep 19 06:15:20.315752 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:20.315721 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8" Sep 19 06:15:20.408644 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:20.408605 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6564aa72-12bb-41e8-9e12-0a88dc25b04d-maas-api-service-ca\") pod \"6564aa72-12bb-41e8-9e12-0a88dc25b04d\" (UID: \"6564aa72-12bb-41e8-9e12-0a88dc25b04d\") " Sep 19 06:15:20.408961 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:20.408932 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6564aa72-12bb-41e8-9e12-0a88dc25b04d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6564aa72-12bb-41e8-9e12-0a88dc25b04d" (UID: "6564aa72-12bb-41e8-9e12-0a88dc25b04d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:15:20.409073 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:20.409041 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6564aa72-12bb-41e8-9e12-0a88dc25b04d-maas-api-service-ca\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:15:21.179102 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:21.179069 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8" Sep 19 06:15:21.179301 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:21.179114 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8" event={"ID":"6564aa72-12bb-41e8-9e12-0a88dc25b04d","Type":"ContainerDied","Data":"f2def852ae6f56efe98183233ee5d44850dc39e01c6b3155424361cacb2208c0"} Sep 19 06:15:21.179301 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:21.179170 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2def852ae6f56efe98183233ee5d44850dc39e01c6b3155424361cacb2208c0" Sep 19 06:15:27.398853 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.398758 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:27.420536 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.420501 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd"] Sep 19 06:15:27.420739 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.420713 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" podUID="2d23ae8c-3653-477e-afa0-08ec44c4c13a" containerName="payload-processing" containerID="cri-o://a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41" gracePeriod=30 Sep 19 06:15:27.427984 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.427951 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99"] Sep 19 06:15:27.428432 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.428146 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" podUID="6b0de816-2634-49c6-94c8-a2c666568195" containerName="payload-pre-processing" containerID="cri-o://3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629" gracePeriod=30 Sep 19 06:15:27.459614 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.459586 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:27.512962 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.512929 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:27.574753 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.574673 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8"] Sep 19 06:15:27.575519 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.575449 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:27.577992 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.577965 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f1aad1-hnsn8"] Sep 19 06:15:27.635847 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.635812 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:27.706285 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.706260 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" Sep 19 06:15:27.709273 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.709251 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" Sep 19 06:15:27.887167 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.887132 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d23ae8c-3653-477e-afa0-08ec44c4c13a-plugins-config-volume\") pod \"2d23ae8c-3653-477e-afa0-08ec44c4c13a\" (UID: \"2d23ae8c-3653-477e-afa0-08ec44c4c13a\") " Sep 19 06:15:27.887379 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.887185 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wz27h\" (UniqueName: \"kubernetes.io/projected/6b0de816-2634-49c6-94c8-a2c666568195-kube-api-access-wz27h\") pod \"6b0de816-2634-49c6-94c8-a2c666568195\" (UID: \"6b0de816-2634-49c6-94c8-a2c666568195\") " Sep 19 06:15:27.887379 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.887229 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz5jm\" (UniqueName: \"kubernetes.io/projected/2d23ae8c-3653-477e-afa0-08ec44c4c13a-kube-api-access-lz5jm\") pod \"2d23ae8c-3653-477e-afa0-08ec44c4c13a\" (UID: \"2d23ae8c-3653-477e-afa0-08ec44c4c13a\") " Sep 19 06:15:27.887379 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.887287 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b0de816-2634-49c6-94c8-a2c666568195-plugins-config-volume\") pod \"6b0de816-2634-49c6-94c8-a2c666568195\" (UID: \"6b0de816-2634-49c6-94c8-a2c666568195\") " Sep 19 06:15:27.887658 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.887629 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d23ae8c-3653-477e-afa0-08ec44c4c13a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2d23ae8c-3653-477e-afa0-08ec44c4c13a" (UID: "2d23ae8c-3653-477e-afa0-08ec44c4c13a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:15:27.887722 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.887698 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b0de816-2634-49c6-94c8-a2c666568195-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b0de816-2634-49c6-94c8-a2c666568195" (UID: "6b0de816-2634-49c6-94c8-a2c666568195"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 06:15:27.889380 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.889355 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d23ae8c-3653-477e-afa0-08ec44c4c13a-kube-api-access-lz5jm" (OuterVolumeSpecName: "kube-api-access-lz5jm") pod "2d23ae8c-3653-477e-afa0-08ec44c4c13a" (UID: "2d23ae8c-3653-477e-afa0-08ec44c4c13a"). InnerVolumeSpecName "kube-api-access-lz5jm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:15:27.889488 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.889381 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b0de816-2634-49c6-94c8-a2c666568195-kube-api-access-wz27h" (OuterVolumeSpecName: "kube-api-access-wz27h") pod "6b0de816-2634-49c6-94c8-a2c666568195" (UID: "6b0de816-2634-49c6-94c8-a2c666568195"). InnerVolumeSpecName "kube-api-access-wz27h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:15:27.969304 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.969205 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:27.988806 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.988764 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b0de816-2634-49c6-94c8-a2c666568195-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:15:27.988806 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.988801 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d23ae8c-3653-477e-afa0-08ec44c4c13a-plugins-config-volume\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:15:27.989013 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.988818 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wz27h\" (UniqueName: \"kubernetes.io/projected/6b0de816-2634-49c6-94c8-a2c666568195-kube-api-access-wz27h\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:15:27.989013 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:27.988837 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz5jm\" (UniqueName: \"kubernetes.io/projected/2d23ae8c-3653-477e-afa0-08ec44c4c13a-kube-api-access-lz5jm\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:15:28.071642 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.071612 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:28.218578 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.218536 2584 generic.go:356] "Generic (PLEG): container finished" podID="6b0de816-2634-49c6-94c8-a2c666568195" containerID="3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629" exitCode=0 Sep 19 06:15:28.218773 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.218586 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" Sep 19 06:15:28.218773 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.218621 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" event={"ID":"6b0de816-2634-49c6-94c8-a2c666568195","Type":"ContainerDied","Data":"3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629"} Sep 19 06:15:28.218773 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.218676 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99" event={"ID":"6b0de816-2634-49c6-94c8-a2c666568195","Type":"ContainerDied","Data":"0fb87eee569a8276acc085f04d7589dc0160a708a5475ffd5e273eb139c3ed0e"} Sep 19 06:15:28.218773 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.218700 2584 scope.go:117] "RemoveContainer" containerID="3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629" Sep 19 06:15:28.220121 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.220101 2584 generic.go:356] "Generic (PLEG): container finished" podID="2d23ae8c-3653-477e-afa0-08ec44c4c13a" containerID="a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41" exitCode=0 Sep 19 06:15:28.220240 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.220149 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" Sep 19 06:15:28.220240 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.220202 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" event={"ID":"2d23ae8c-3653-477e-afa0-08ec44c4c13a","Type":"ContainerDied","Data":"a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41"} Sep 19 06:15:28.220346 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.220260 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd" event={"ID":"2d23ae8c-3653-477e-afa0-08ec44c4c13a","Type":"ContainerDied","Data":"07161157b1ef73a91d9a0fe72716059c53a0f7d6d494a79682c0ac28c5133f16"} Sep 19 06:15:28.229444 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.229419 2584 scope.go:117] "RemoveContainer" containerID="3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629" Sep 19 06:15:28.229758 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:15:28.229738 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629\": container with ID starting with 3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629 not found: ID does not exist" containerID="3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629" Sep 19 06:15:28.229826 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.229767 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629"} err="failed to get container status \"3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629\": rpc error: code = NotFound desc = could not find container \"3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629\": container with ID starting with 3d1e8b7092024d6e0515c94a78ea99d1479ac952f3564ef847308d1d0f05c629 not found: ID does not exist" Sep 19 06:15:28.229826 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.229785 2584 scope.go:117] "RemoveContainer" containerID="a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41" Sep 19 06:15:28.238733 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.238705 2584 scope.go:117] "RemoveContainer" containerID="a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41" Sep 19 06:15:28.239016 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:15:28.238995 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41\": container with ID starting with a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41 not found: ID does not exist" containerID="a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41" Sep 19 06:15:28.239067 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.239025 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41"} err="failed to get container status \"a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41\": rpc error: code = NotFound desc = could not find container \"a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41\": container with ID starting with a15edc89abc8482946ace34749636704d79e66ce4a14e7f39318ef2afcd5bb41 not found: ID does not exist" Sep 19 06:15:28.243273 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.243248 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd"] Sep 19 06:15:28.247039 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.247018 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-f1aad1-b959bf785-2b8rd"] Sep 19 06:15:28.259500 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.259470 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99"] Sep 19 06:15:28.268099 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.268074 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d23ae8c-3653-477e-afa0-08ec44c4c13a" path="/var/lib/kubelet/pods/2d23ae8c-3653-477e-afa0-08ec44c4c13a/volumes" Sep 19 06:15:28.268457 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.268443 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6564aa72-12bb-41e8-9e12-0a88dc25b04d" path="/var/lib/kubelet/pods/6564aa72-12bb-41e8-9e12-0a88dc25b04d/volumes" Sep 19 06:15:28.268720 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:28.268706 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-f1aad1-6f5fc4d8d6-wpq99"] Sep 19 06:15:30.267358 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:30.267321 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b0de816-2634-49c6-94c8-a2c666568195" path="/var/lib/kubelet/pods/6b0de816-2634-49c6-94c8-a2c666568195/volumes" Sep 19 06:15:33.765704 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:33.765667 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:33.825785 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:33.825755 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:34.027308 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:34.027222 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:15:37.031017 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.030977 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-687fc9b78d-xgndz"] Sep 19 06:15:37.031514 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.031183 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-687fc9b78d-xgndz" podUID="b23c027b-c65f-49aa-82c3-834df63718e5" containerName="manager" containerID="cri-o://6529a2afa91f82238a41c9d8ea7a8620c803a2837f87a0b547b9a8fcd449ef9e" gracePeriod=30 Sep 19 06:15:37.274280 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.274251 2584 generic.go:356] "Generic (PLEG): container finished" podID="b23c027b-c65f-49aa-82c3-834df63718e5" containerID="6529a2afa91f82238a41c9d8ea7a8620c803a2837f87a0b547b9a8fcd449ef9e" exitCode=0 Sep 19 06:15:37.274423 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.274289 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687fc9b78d-xgndz" event={"ID":"b23c027b-c65f-49aa-82c3-834df63718e5","Type":"ContainerDied","Data":"6529a2afa91f82238a41c9d8ea7a8620c803a2837f87a0b547b9a8fcd449ef9e"} Sep 19 06:15:37.274423 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.274327 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687fc9b78d-xgndz" event={"ID":"b23c027b-c65f-49aa-82c3-834df63718e5","Type":"ContainerDied","Data":"376577728b80c69b88a706dc53b85c88c5c579313d4f2b3f36f39ad82ae2f88b"} Sep 19 06:15:37.274423 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.274342 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="376577728b80c69b88a706dc53b85c88c5c579313d4f2b3f36f39ad82ae2f88b" Sep 19 06:15:37.285024 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.284965 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 06:15:37.368794 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.368753 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-metrics-certs\") pod \"b23c027b-c65f-49aa-82c3-834df63718e5\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " Sep 19 06:15:37.368992 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.368818 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-webhook-cert\") pod \"b23c027b-c65f-49aa-82c3-834df63718e5\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " Sep 19 06:15:37.368992 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.368887 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kt9mv\" (UniqueName: \"kubernetes.io/projected/b23c027b-c65f-49aa-82c3-834df63718e5-kube-api-access-kt9mv\") pod \"b23c027b-c65f-49aa-82c3-834df63718e5\" (UID: \"b23c027b-c65f-49aa-82c3-834df63718e5\") " Sep 19 06:15:37.370984 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.370957 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b23c027b-c65f-49aa-82c3-834df63718e5" (UID: "b23c027b-c65f-49aa-82c3-834df63718e5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 06:15:37.370984 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.370972 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b23c027b-c65f-49aa-82c3-834df63718e5-kube-api-access-kt9mv" (OuterVolumeSpecName: "kube-api-access-kt9mv") pod "b23c027b-c65f-49aa-82c3-834df63718e5" (UID: "b23c027b-c65f-49aa-82c3-834df63718e5"). InnerVolumeSpecName "kube-api-access-kt9mv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:15:37.370984 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.370976 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b23c027b-c65f-49aa-82c3-834df63718e5" (UID: "b23c027b-c65f-49aa-82c3-834df63718e5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 06:15:37.470182 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.470132 2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-metrics-certs\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:15:37.470182 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.470171 2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b23c027b-c65f-49aa-82c3-834df63718e5-webhook-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:15:37.470182 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:37.470183 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt9mv\" (UniqueName: \"kubernetes.io/projected/b23c027b-c65f-49aa-82c3-834df63718e5-kube-api-access-kt9mv\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:15:38.279195 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:38.279166 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687fc9b78d-xgndz" Sep 19 06:15:38.294512 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:38.294482 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-687fc9b78d-xgndz"] Sep 19 06:15:38.297501 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:38.297476 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-687fc9b78d-xgndz"] Sep 19 06:15:40.267599 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:40.267566 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b23c027b-c65f-49aa-82c3-834df63718e5" path="/var/lib/kubelet/pods/b23c027b-c65f-49aa-82c3-834df63718e5/volumes" Sep 19 06:15:43.120735 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:15:43.120699 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:16:14.419217 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:14.419187 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 06:16:14.423305 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:14.423278 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 06:16:14.903613 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:14.903584 2584 scope.go:117] "RemoveContainer" containerID="6529a2afa91f82238a41c9d8ea7a8620c803a2837f87a0b547b9a8fcd449ef9e" Sep 19 06:16:39.110468 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110422 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-687fc9b78d-dt472"] Sep 19 06:16:39.112979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110819 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6564aa72-12bb-41e8-9e12-0a88dc25b04d" containerName="revoke-keys" Sep 19 06:16:39.112979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110830 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6564aa72-12bb-41e8-9e12-0a88dc25b04d" containerName="revoke-keys" Sep 19 06:16:39.112979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110838 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b0de816-2634-49c6-94c8-a2c666568195" containerName="payload-pre-processing" Sep 19 06:16:39.112979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110844 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b0de816-2634-49c6-94c8-a2c666568195" containerName="payload-pre-processing" Sep 19 06:16:39.112979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110866 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b23c027b-c65f-49aa-82c3-834df63718e5" containerName="manager" Sep 19 06:16:39.112979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110872 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b23c027b-c65f-49aa-82c3-834df63718e5" containerName="manager" Sep 19 06:16:39.112979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110890 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d23ae8c-3653-477e-afa0-08ec44c4c13a" containerName="payload-processing" Sep 19 06:16:39.112979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110896 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d23ae8c-3653-477e-afa0-08ec44c4c13a" containerName="payload-processing" Sep 19 06:16:39.112979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110960 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6564aa72-12bb-41e8-9e12-0a88dc25b04d" containerName="revoke-keys" Sep 19 06:16:39.112979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110968 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d23ae8c-3653-477e-afa0-08ec44c4c13a" containerName="payload-processing" Sep 19 06:16:39.112979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110976 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b0de816-2634-49c6-94c8-a2c666568195" containerName="payload-pre-processing" Sep 19 06:16:39.112979 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.110983 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b23c027b-c65f-49aa-82c3-834df63718e5" containerName="manager" Sep 19 06:16:39.114044 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.114027 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:16:39.117598 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.117568 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 19 06:16:39.117598 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.117577 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 19 06:16:39.117757 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.117625 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 19 06:16:39.117757 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.117597 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-znct9\"" Sep 19 06:16:39.122293 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.122271 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-687fc9b78d-dt472"] Sep 19 06:16:39.236807 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.236769 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-webhook-cert\") pod \"maas-controller-687fc9b78d-dt472\" (UID: \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\") " pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:16:39.237015 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.236906 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-metrics-certs\") pod \"maas-controller-687fc9b78d-dt472\" (UID: \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\") " pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:16:39.237015 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.236950 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv7cg\" (UniqueName: \"kubernetes.io/projected/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-kube-api-access-qv7cg\") pod \"maas-controller-687fc9b78d-dt472\" (UID: \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\") " pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:16:39.337983 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.337937 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-webhook-cert\") pod \"maas-controller-687fc9b78d-dt472\" (UID: \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\") " pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:16:39.338199 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.338053 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-metrics-certs\") pod \"maas-controller-687fc9b78d-dt472\" (UID: \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\") " pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:16:39.338199 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.338088 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv7cg\" (UniqueName: \"kubernetes.io/projected/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-kube-api-access-qv7cg\") pod \"maas-controller-687fc9b78d-dt472\" (UID: \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\") " pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:16:39.340608 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.340576 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-webhook-cert\") pod \"maas-controller-687fc9b78d-dt472\" (UID: \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\") " pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:16:39.340730 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.340631 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-metrics-certs\") pod \"maas-controller-687fc9b78d-dt472\" (UID: \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\") " pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:16:39.345721 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.345697 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv7cg\" (UniqueName: \"kubernetes.io/projected/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-kube-api-access-qv7cg\") pod \"maas-controller-687fc9b78d-dt472\" (UID: \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\") " pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:16:39.426311 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.426206 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:16:39.697030 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.697004 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-687fc9b78d-dt472"] Sep 19 06:16:39.699926 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:16:39.699903 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611f2ae4_31aa_4e04_96c9_69e04a9f23e3.slice/crio-f93170270b69cda7e6a0f92f900059ff1f2ff57e0d8ed16a49ab07d52a6cd7ac WatchSource:0}: Error finding container f93170270b69cda7e6a0f92f900059ff1f2ff57e0d8ed16a49ab07d52a6cd7ac: Status 404 returned error can't find the container with id f93170270b69cda7e6a0f92f900059ff1f2ff57e0d8ed16a49ab07d52a6cd7ac Sep 19 06:16:39.701268 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:39.701250 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 06:16:40.619638 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:40.619603 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687fc9b78d-dt472" event={"ID":"611f2ae4-31aa-4e04-96c9-69e04a9f23e3","Type":"ContainerStarted","Data":"bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156"} Sep 19 06:16:40.620080 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:40.619657 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687fc9b78d-dt472" event={"ID":"611f2ae4-31aa-4e04-96c9-69e04a9f23e3","Type":"ContainerStarted","Data":"f93170270b69cda7e6a0f92f900059ff1f2ff57e0d8ed16a49ab07d52a6cd7ac"} Sep 19 06:16:40.620080 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:40.619702 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:16:40.638677 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:40.638627 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-687fc9b78d-dt472" podStartSLOduration=1.12423144 podStartE2EDuration="1.638611703s" totalImagesPullingTime="514.380263ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:16:40.636856365 +0000 UTC m=+2128.630083896" watchObservedRunningTime="2026-09-19 06:16:40.638611703 +0000 UTC m=+2128.631839233" Sep 19 06:16:51.627322 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:16:51.627286 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:17:02.993345 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:02.993272 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:17:03.092271 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.092230 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-687fc9b78d-dt472"] Sep 19 06:17:03.092588 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.092545 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-687fc9b78d-dt472" podUID="611f2ae4-31aa-4e04-96c9-69e04a9f23e3" containerName="manager" containerID="cri-o://bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156" gracePeriod=30 Sep 19 06:17:03.340680 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.340646 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:17:03.378645 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.378604 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-metrics-certs\") pod \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\" (UID: \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\") " Sep 19 06:17:03.378838 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.378679 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-webhook-cert\") pod \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\" (UID: \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\") " Sep 19 06:17:03.378838 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.378718 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qv7cg\" (UniqueName: \"kubernetes.io/projected/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-kube-api-access-qv7cg\") pod \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\" (UID: \"611f2ae4-31aa-4e04-96c9-69e04a9f23e3\") " Sep 19 06:17:03.381144 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.381090 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-kube-api-access-qv7cg" (OuterVolumeSpecName: "kube-api-access-qv7cg") pod "611f2ae4-31aa-4e04-96c9-69e04a9f23e3" (UID: "611f2ae4-31aa-4e04-96c9-69e04a9f23e3"). InnerVolumeSpecName "kube-api-access-qv7cg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 06:17:03.381378 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.381351 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "611f2ae4-31aa-4e04-96c9-69e04a9f23e3" (UID: "611f2ae4-31aa-4e04-96c9-69e04a9f23e3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 06:17:03.381531 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.381379 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "611f2ae4-31aa-4e04-96c9-69e04a9f23e3" (UID: "611f2ae4-31aa-4e04-96c9-69e04a9f23e3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 06:17:03.480011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.479967 2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-metrics-certs\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:17:03.480011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.480000 2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-webhook-cert\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:17:03.480011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.480009 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv7cg\" (UniqueName: \"kubernetes.io/projected/611f2ae4-31aa-4e04-96c9-69e04a9f23e3-kube-api-access-qv7cg\") on node \"ip-10-0-142-57.ec2.internal\" DevicePath \"\"" Sep 19 06:17:03.759766 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.759732 2584 generic.go:356] "Generic (PLEG): container finished" podID="611f2ae4-31aa-4e04-96c9-69e04a9f23e3" containerID="bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156" exitCode=0 Sep 19 06:17:03.759953 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.759773 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687fc9b78d-dt472" Sep 19 06:17:03.759953 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.759808 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687fc9b78d-dt472" event={"ID":"611f2ae4-31aa-4e04-96c9-69e04a9f23e3","Type":"ContainerDied","Data":"bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156"} Sep 19 06:17:03.759953 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.759864 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687fc9b78d-dt472" event={"ID":"611f2ae4-31aa-4e04-96c9-69e04a9f23e3","Type":"ContainerDied","Data":"f93170270b69cda7e6a0f92f900059ff1f2ff57e0d8ed16a49ab07d52a6cd7ac"} Sep 19 06:17:03.759953 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.759885 2584 scope.go:117] "RemoveContainer" containerID="bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156" Sep 19 06:17:03.769576 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.769552 2584 scope.go:117] "RemoveContainer" containerID="bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156" Sep 19 06:17:03.769924 ip-10-0-142-57 kubenswrapper[2584]: E0919 06:17:03.769902 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156\": container with ID starting with bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156 not found: ID does not exist" containerID="bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156" Sep 19 06:17:03.769968 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.769937 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156"} err="failed to get container status \"bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156\": rpc error: code = NotFound desc = could not find container \"bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156\": container with ID starting with bdd1531a940d7b46e267556782e74737aa4c318f97b253ea87b08439e45dd156 not found: ID does not exist" Sep 19 06:17:03.781224 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.781195 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-687fc9b78d-dt472"] Sep 19 06:17:03.784294 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:03.784265 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-687fc9b78d-dt472"] Sep 19 06:17:04.269799 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.269766 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="611f2ae4-31aa-4e04-96c9-69e04a9f23e3" path="/var/lib/kubelet/pods/611f2ae4-31aa-4e04-96c9-69e04a9f23e3/volumes" Sep 19 06:17:04.463759 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.463727 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:17:04.577879 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.577797 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-687fc9b78d-48kkf"] Sep 19 06:17:04.578230 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.578217 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="611f2ae4-31aa-4e04-96c9-69e04a9f23e3" containerName="manager" Sep 19 06:17:04.578277 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.578231 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="611f2ae4-31aa-4e04-96c9-69e04a9f23e3" containerName="manager" Sep 19 06:17:04.578317 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.578295 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="611f2ae4-31aa-4e04-96c9-69e04a9f23e3" containerName="manager" Sep 19 06:17:04.583198 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.583178 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:04.586698 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.586670 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 19 06:17:04.586946 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.586706 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 19 06:17:04.586946 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.586762 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 19 06:17:04.586946 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.586682 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-znct9\"" Sep 19 06:17:04.589071 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.589048 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-687fc9b78d-48kkf"] Sep 19 06:17:04.692414 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.692369 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97a3601f-1d00-4289-9c7f-d759b0d552fb-webhook-cert\") pod \"maas-controller-687fc9b78d-48kkf\" (UID: \"97a3601f-1d00-4289-9c7f-d759b0d552fb\") " pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:04.692621 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.692440 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97a3601f-1d00-4289-9c7f-d759b0d552fb-metrics-certs\") pod \"maas-controller-687fc9b78d-48kkf\" (UID: \"97a3601f-1d00-4289-9c7f-d759b0d552fb\") " pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:04.692621 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.692567 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v7qz\" (UniqueName: \"kubernetes.io/projected/97a3601f-1d00-4289-9c7f-d759b0d552fb-kube-api-access-4v7qz\") pod \"maas-controller-687fc9b78d-48kkf\" (UID: \"97a3601f-1d00-4289-9c7f-d759b0d552fb\") " pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:04.794085 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.794039 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97a3601f-1d00-4289-9c7f-d759b0d552fb-webhook-cert\") pod \"maas-controller-687fc9b78d-48kkf\" (UID: \"97a3601f-1d00-4289-9c7f-d759b0d552fb\") " pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:04.794085 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.794084 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97a3601f-1d00-4289-9c7f-d759b0d552fb-metrics-certs\") pod \"maas-controller-687fc9b78d-48kkf\" (UID: \"97a3601f-1d00-4289-9c7f-d759b0d552fb\") " pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:04.794356 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.794182 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v7qz\" (UniqueName: \"kubernetes.io/projected/97a3601f-1d00-4289-9c7f-d759b0d552fb-kube-api-access-4v7qz\") pod \"maas-controller-687fc9b78d-48kkf\" (UID: \"97a3601f-1d00-4289-9c7f-d759b0d552fb\") " pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:04.796640 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.796607 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97a3601f-1d00-4289-9c7f-d759b0d552fb-webhook-cert\") pod \"maas-controller-687fc9b78d-48kkf\" (UID: \"97a3601f-1d00-4289-9c7f-d759b0d552fb\") " pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:04.796786 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.796712 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97a3601f-1d00-4289-9c7f-d759b0d552fb-metrics-certs\") pod \"maas-controller-687fc9b78d-48kkf\" (UID: \"97a3601f-1d00-4289-9c7f-d759b0d552fb\") " pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:04.801262 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.801237 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v7qz\" (UniqueName: \"kubernetes.io/projected/97a3601f-1d00-4289-9c7f-d759b0d552fb-kube-api-access-4v7qz\") pod \"maas-controller-687fc9b78d-48kkf\" (UID: \"97a3601f-1d00-4289-9c7f-d759b0d552fb\") " pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:04.895728 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:04.895692 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:05.165754 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:05.165725 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-687fc9b78d-48kkf"] Sep 19 06:17:05.168044 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:17:05.168015 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97a3601f_1d00_4289_9c7f_d759b0d552fb.slice/crio-3ec26905e570fb14d3b914e34fd3f679e7c25b07be64c9a5a6c614551e282f4a WatchSource:0}: Error finding container 3ec26905e570fb14d3b914e34fd3f679e7c25b07be64c9a5a6c614551e282f4a: Status 404 returned error can't find the container with id 3ec26905e570fb14d3b914e34fd3f679e7c25b07be64c9a5a6c614551e282f4a Sep 19 06:17:05.284232 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:05.284198 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:17:05.385324 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:05.385291 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:17:05.773351 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:05.773258 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687fc9b78d-48kkf" event={"ID":"97a3601f-1d00-4289-9c7f-d759b0d552fb","Type":"ContainerStarted","Data":"756f433b311f9dc25b914569ad8c945ce8895b3021747c5e45e87ec81690e000"} Sep 19 06:17:05.773351 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:05.773311 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-687fc9b78d-48kkf" event={"ID":"97a3601f-1d00-4289-9c7f-d759b0d552fb","Type":"ContainerStarted","Data":"3ec26905e570fb14d3b914e34fd3f679e7c25b07be64c9a5a6c614551e282f4a"} Sep 19 06:17:05.773351 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:05.773345 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:05.789569 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:05.789498 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-687fc9b78d-48kkf" podStartSLOduration=1.464048615 podStartE2EDuration="1.789481091s" totalImagesPullingTime="325.432476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:17:05.788050722 +0000 UTC m=+2153.781278256" watchObservedRunningTime="2026-09-19 06:17:05.789481091 +0000 UTC m=+2153.782708621" Sep 19 06:17:16.782485 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:16.782453 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-687fc9b78d-48kkf" Sep 19 06:17:57.373960 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:17:57.373920 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:18:22.226210 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:18:22.226170 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:18:22.275680 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:18:22.275647 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:19:41.812626 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:19:41.812593 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:19:44.678856 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:19:44.678812 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:19:44.727754 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:19:44.727706 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:20:14.961192 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:20:14.961151 2584 scope.go:117] "RemoveContainer" containerID="8183e36d1de5572b359809fe9280da72c0a7a37c56fbf26a7045a9f573d36975" Sep 19 06:20:14.970134 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:20:14.970107 2584 scope.go:117] "RemoveContainer" containerID="1ea9f051c350e095426bc6809860690b13ea5ff709fd7e58e98d4e4b1ddb473b" Sep 19 06:20:28.763654 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:20:28.763616 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:20:32.115857 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:20:32.115823 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:20:32.215977 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:20:32.215943 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:20:34.063034 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:20:34.063003 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:20:39.749014 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:20:39.748981 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:20:47.772362 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:20:47.772326 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:21:14.106461 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:21:14.106425 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:21:14.453889 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:21:14.453808 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 06:21:14.459931 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:21:14.459908 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzjr9_795198a1-531e-4263-89f7-59d4a963ef34/ovn-acl-logging/0.log" Sep 19 06:21:14.646006 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:21:14.645966 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:21:14.994593 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:21:14.994538 2584 scope.go:117] "RemoveContainer" containerID="8d5b5a32e667aaf8aa3089bbf78f857d4a7a5c6ada0caa852f9501cdcaa0d5a0" Sep 19 06:21:15.003224 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:21:15.003199 2584 scope.go:117] "RemoveContainer" containerID="4d31474cfef0e7562a7a1a2c37b720628a9aff2507868e1b8afbd78e3fb4f0b3" Sep 19 06:21:15.011264 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:21:15.011244 2584 scope.go:117] "RemoveContainer" containerID="c8fec2aef3159cc21a33cc99994ec436f2afc7ecd6de284ea57ee0ef2d6d0a83" Sep 19 06:22:03.768859 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:03.768824 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5d6965647f-c5b7z_6904f775-503e-4110-9747-87165a60ad3d/authorino/0.log" Sep 19 06:22:09.449736 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:09.449701 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-687fc9b78d-48kkf_97a3601f-1d00-4289-9c7f-d759b0d552fb/manager/0.log" Sep 19 06:22:09.945176 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:09.945150 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fd6794c5b-ht29j_732f2a56-951f-4a7b-a3b7-2be38ff1c869/copy-manifests/0.log" Sep 19 06:22:09.951619 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:09.951593 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fd6794c5b-ht29j_732f2a56-951f-4a7b-a3b7-2be38ff1c869/manager/0.log" Sep 19 06:22:10.642978 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:10.642954 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29829960-fx5mq_07bab225-a9be-4379-97e4-aa04e662c76a/cleanup/0.log" Sep 19 06:22:10.869728 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:10.869699 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-rzc2q_ebfa06ac-808c-4082-a57d-f287453b462f/postgres/0.log" Sep 19 06:22:11.658328 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:11.658289 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5d6965647f-c5b7z_6904f775-503e-4110-9747-87165a60ad3d/authorino/0.log" Sep 19 06:22:13.095992 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:13.095962 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b984b98c7-r7tgg_6e631cb4-6f0e-4c0f-9350-8aaff40a84f9/kube-auth-proxy/0.log" Sep 19 06:22:13.340559 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:13.340524 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-spp4h_2c4703ef-5647-4533-a88f-6443cace6aec/payload-pre-processing/0.log" Sep 19 06:22:13.462011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:13.461922 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dmnts_e3c2b6d2-b60c-4dde-906b-7301fea6c7f5/payload-processing/0.log" Sep 19 06:22:13.934799 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:13.934770 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-l4q2d_1356cd98-57e2-4fc7-acf0-4ccfa5e92e77/main/0.log" Sep 19 06:22:14.278036 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:14.277959 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-rd2mg_ff5dba78-1de8-48a7-a66c-060b364024c2/main/0.log" Sep 19 06:22:15.037753 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:15.037723 2584 scope.go:117] "RemoveContainer" containerID="2e7c9e3a8bbd3fc32949f9bd39698b033e383bfe96fe300d4aaba1c02b65b71b" Sep 19 06:22:20.153826 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.153789 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:22:20.200091 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.200054 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5n4fs/must-gather-ll57p"] Sep 19 06:22:20.205429 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.205393 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5n4fs/must-gather-ll57p" Sep 19 06:22:20.207807 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.207777 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5n4fs\"/\"openshift-service-ca.crt\"" Sep 19 06:22:20.207992 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.207969 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5n4fs\"/\"kube-root-ca.crt\"" Sep 19 06:22:20.208071 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.208010 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5n4fs\"/\"default-dockercfg-96wgx\"" Sep 19 06:22:20.210774 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.210749 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5n4fs/must-gather-ll57p"] Sep 19 06:22:20.216547 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.216515 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" limit=1024 Sep 19 06:22:20.265626 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.265586 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ee72c264-130f-408c-bb9c-45531aec199e-must-gather-output\") pod \"must-gather-ll57p\" (UID: \"ee72c264-130f-408c-bb9c-45531aec199e\") " pod="openshift-must-gather-5n4fs/must-gather-ll57p" Sep 19 06:22:20.265626 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.265618 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jxfv\" (UniqueName: \"kubernetes.io/projected/ee72c264-130f-408c-bb9c-45531aec199e-kube-api-access-2jxfv\") pod \"must-gather-ll57p\" (UID: \"ee72c264-130f-408c-bb9c-45531aec199e\") " pod="openshift-must-gather-5n4fs/must-gather-ll57p" Sep 19 06:22:20.366496 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.366459 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ee72c264-130f-408c-bb9c-45531aec199e-must-gather-output\") pod \"must-gather-ll57p\" (UID: \"ee72c264-130f-408c-bb9c-45531aec199e\") " pod="openshift-must-gather-5n4fs/must-gather-ll57p" Sep 19 06:22:20.366496 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.366498 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxfv\" (UniqueName: \"kubernetes.io/projected/ee72c264-130f-408c-bb9c-45531aec199e-kube-api-access-2jxfv\") pod \"must-gather-ll57p\" (UID: \"ee72c264-130f-408c-bb9c-45531aec199e\") " pod="openshift-must-gather-5n4fs/must-gather-ll57p" Sep 19 06:22:20.366825 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.366804 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ee72c264-130f-408c-bb9c-45531aec199e-must-gather-output\") pod \"must-gather-ll57p\" (UID: \"ee72c264-130f-408c-bb9c-45531aec199e\") " pod="openshift-must-gather-5n4fs/must-gather-ll57p" Sep 19 06:22:20.374351 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.374331 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jxfv\" (UniqueName: \"kubernetes.io/projected/ee72c264-130f-408c-bb9c-45531aec199e-kube-api-access-2jxfv\") pod \"must-gather-ll57p\" (UID: \"ee72c264-130f-408c-bb9c-45531aec199e\") " pod="openshift-must-gather-5n4fs/must-gather-ll57p" Sep 19 06:22:20.516227 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.516133 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5n4fs/must-gather-ll57p" Sep 19 06:22:20.790281 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.790253 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5n4fs/must-gather-ll57p"] Sep 19 06:22:20.792519 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:22:20.792489 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee72c264_130f_408c_bb9c_45531aec199e.slice/crio-268a206b8d860cb1d750d07eb9a38ebc8e4cf3ef3495961bf33a9ed813fef274 WatchSource:0}: Error finding container 268a206b8d860cb1d750d07eb9a38ebc8e4cf3ef3495961bf33a9ed813fef274: Status 404 returned error can't find the container with id 268a206b8d860cb1d750d07eb9a38ebc8e4cf3ef3495961bf33a9ed813fef274 Sep 19 06:22:20.794458 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:20.794435 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 06:22:21.456793 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:21.456752 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5n4fs/must-gather-ll57p" event={"ID":"ee72c264-130f-408c-bb9c-45531aec199e","Type":"ContainerStarted","Data":"268a206b8d860cb1d750d07eb9a38ebc8e4cf3ef3495961bf33a9ed813fef274"} Sep 19 06:22:22.467595 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:22.467554 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5n4fs/must-gather-ll57p" event={"ID":"ee72c264-130f-408c-bb9c-45531aec199e","Type":"ContainerStarted","Data":"4195192be795dbf1b39130062f22707b75637869b42b045f8a5ec5808fce89d0"} Sep 19 06:22:22.467978 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:22.467612 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5n4fs/must-gather-ll57p" event={"ID":"ee72c264-130f-408c-bb9c-45531aec199e","Type":"ContainerStarted","Data":"2813ca6060ea07cd8918a39919aaf0a14223dc5a967b0dadbb09e7940cc078a9"} Sep 19 06:22:22.483370 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:22.482255 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5n4fs/must-gather-ll57p" podStartSLOduration=1.721983243 podStartE2EDuration="2.482235416s" totalImagesPullingTime="760.252173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 06:22:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:22:22.48108122 +0000 UTC m=+2470.474308780" watchObservedRunningTime="2026-09-19 06:22:22.482235416 +0000 UTC m=+2470.475462948" Sep 19 06:22:23.164791 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:23.164731 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-s9kmn_8d49d458-d0eb-4619-9b41-2aa12b664541/global-pull-secret-syncer/0.log" Sep 19 06:22:23.212936 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:23.212898 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7jfqn_f34c569d-7d66-4101-be51-00dbcc995a30/konnectivity-agent/0.log" Sep 19 06:22:23.345911 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:23.345881 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-57.ec2.internal_f782f89f5d636e4550fa0cef2186e194/haproxy/0.log" Sep 19 06:22:28.247390 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:28.247346 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5d6965647f-c5b7z_6904f775-503e-4110-9747-87165a60ad3d/authorino/0.log" Sep 19 06:22:29.806966 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:29.806934 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bd88ec05-cf6b-4480-8469-d997b0453cb8/alertmanager/0.log" Sep 19 06:22:29.828974 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:29.828942 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bd88ec05-cf6b-4480-8469-d997b0453cb8/config-reloader/0.log" Sep 19 06:22:29.850251 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:29.850219 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bd88ec05-cf6b-4480-8469-d997b0453cb8/kube-rbac-proxy-web/0.log" Sep 19 06:22:29.871682 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:29.871650 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bd88ec05-cf6b-4480-8469-d997b0453cb8/kube-rbac-proxy/0.log" Sep 19 06:22:29.890739 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:29.890708 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bd88ec05-cf6b-4480-8469-d997b0453cb8/kube-rbac-proxy-metric/0.log" Sep 19 06:22:29.915661 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:29.915630 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bd88ec05-cf6b-4480-8469-d997b0453cb8/prom-label-proxy/0.log" Sep 19 06:22:29.936753 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:29.936714 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bd88ec05-cf6b-4480-8469-d997b0453cb8/init-config-reloader/0.log" Sep 19 06:22:30.014205 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.014172 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-s4gwv_07b5c728-7106-4bab-87f1-007e05822927/kube-state-metrics/0.log" Sep 19 06:22:30.035011 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.034977 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-s4gwv_07b5c728-7106-4bab-87f1-007e05822927/kube-rbac-proxy-main/0.log" Sep 19 06:22:30.055906 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.055758 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-s4gwv_07b5c728-7106-4bab-87f1-007e05822927/kube-rbac-proxy-self/0.log" Sep 19 06:22:30.086424 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.085981 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6b46b4776c-8zmtc_7c778cac-fa09-42dd-966c-8dd75ec2e1b8/metrics-server/0.log" Sep 19 06:22:30.112590 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.112556 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-7c4js_0d5ae841-0fb7-4036-81a7-8619412a5e3f/monitoring-plugin/0.log" Sep 19 06:22:30.147444 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.147393 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cjrpq_e87e7c2f-8946-40ef-8198-2daac6a33144/node-exporter/0.log" Sep 19 06:22:30.169457 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.169426 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cjrpq_e87e7c2f-8946-40ef-8198-2daac6a33144/kube-rbac-proxy/0.log" Sep 19 06:22:30.194140 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.194105 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cjrpq_e87e7c2f-8946-40ef-8198-2daac6a33144/init-textfile/0.log" Sep 19 06:22:30.412421 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.411581 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-r8tjl_3335bcd0-e181-42c7-a694-2191e06d524c/kube-rbac-proxy-main/0.log" Sep 19 06:22:30.441420 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.440771 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-r8tjl_3335bcd0-e181-42c7-a694-2191e06d524c/kube-rbac-proxy-self/0.log" Sep 19 06:22:30.469418 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.469093 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-r8tjl_3335bcd0-e181-42c7-a694-2191e06d524c/openshift-state-metrics/0.log" Sep 19 06:22:30.509537 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.509510 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbf49520-2af6-4f63-97ce-35d666e85062/prometheus/0.log" Sep 19 06:22:30.531096 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.531066 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbf49520-2af6-4f63-97ce-35d666e85062/config-reloader/0.log" Sep 19 06:22:30.552739 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.552704 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbf49520-2af6-4f63-97ce-35d666e85062/thanos-sidecar/0.log" Sep 19 06:22:30.572936 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.572851 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbf49520-2af6-4f63-97ce-35d666e85062/kube-rbac-proxy-web/0.log" Sep 19 06:22:30.593847 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.593815 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbf49520-2af6-4f63-97ce-35d666e85062/kube-rbac-proxy/0.log" Sep 19 06:22:30.617451 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.617331 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbf49520-2af6-4f63-97ce-35d666e85062/kube-rbac-proxy-thanos/0.log" Sep 19 06:22:30.638904 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.638868 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dbf49520-2af6-4f63-97ce-35d666e85062/init-config-reloader/0.log" Sep 19 06:22:30.671743 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.671690 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-s7q4s_e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f/prometheus-operator/0.log" Sep 19 06:22:30.688085 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.688058 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-s7q4s_e6bc30ff-ee2d-4d45-8d42-db7a6a6e842f/kube-rbac-proxy/0.log" Sep 19 06:22:30.724536 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.724498 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-5krs8_25b751f3-4663-4f33-9170-ebeb996374c8/prometheus-operator-admission-webhook/0.log" Sep 19 06:22:30.754179 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.754149 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86b9b79cb5-g7fvk_961b15e0-e8e1-429a-8f40-33ad81888eb4/telemeter-client/0.log" Sep 19 06:22:30.774142 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.774099 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86b9b79cb5-g7fvk_961b15e0-e8e1-429a-8f40-33ad81888eb4/reload/0.log" Sep 19 06:22:30.800093 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.800061 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86b9b79cb5-g7fvk_961b15e0-e8e1-429a-8f40-33ad81888eb4/kube-rbac-proxy/0.log" Sep 19 06:22:30.833429 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.833376 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75f8b449ff-l4x2r_67f88c7a-c35f-46cb-83f6-2ce328bc297f/thanos-query/0.log" Sep 19 06:22:30.854533 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.854506 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75f8b449ff-l4x2r_67f88c7a-c35f-46cb-83f6-2ce328bc297f/kube-rbac-proxy-web/0.log" Sep 19 06:22:30.879044 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.878989 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75f8b449ff-l4x2r_67f88c7a-c35f-46cb-83f6-2ce328bc297f/kube-rbac-proxy/0.log" Sep 19 06:22:30.899921 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.899881 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75f8b449ff-l4x2r_67f88c7a-c35f-46cb-83f6-2ce328bc297f/prom-label-proxy/0.log" Sep 19 06:22:30.930351 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.930296 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75f8b449ff-l4x2r_67f88c7a-c35f-46cb-83f6-2ce328bc297f/kube-rbac-proxy-rules/0.log" Sep 19 06:22:30.961781 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:30.961749 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75f8b449ff-l4x2r_67f88c7a-c35f-46cb-83f6-2ce328bc297f/kube-rbac-proxy-metrics/0.log" Sep 19 06:22:33.535705 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.535642 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5bc6d7896d-j64jf_ed7d33d3-a4dd-4a1b-bedc-34bfb6298d83/console/0.log" Sep 19 06:22:33.698118 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.698074 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m"] Sep 19 06:22:33.706028 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.705995 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.710866 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.710802 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m"] Sep 19 06:22:33.822389 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.822279 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdpkd\" (UniqueName: \"kubernetes.io/projected/c58059a7-6967-4254-b7e7-ca566eb3de87-kube-api-access-zdpkd\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.822389 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.822384 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c58059a7-6967-4254-b7e7-ca566eb3de87-proc\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.822693 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.822529 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c58059a7-6967-4254-b7e7-ca566eb3de87-podres\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.822693 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.822574 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c58059a7-6967-4254-b7e7-ca566eb3de87-lib-modules\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.822693 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.822645 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c58059a7-6967-4254-b7e7-ca566eb3de87-sys\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.924075 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.924033 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c58059a7-6967-4254-b7e7-ca566eb3de87-sys\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.924274 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.924121 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdpkd\" (UniqueName: \"kubernetes.io/projected/c58059a7-6967-4254-b7e7-ca566eb3de87-kube-api-access-zdpkd\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.924274 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.924151 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c58059a7-6967-4254-b7e7-ca566eb3de87-proc\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.924274 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.924154 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c58059a7-6967-4254-b7e7-ca566eb3de87-sys\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.924274 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.924233 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c58059a7-6967-4254-b7e7-ca566eb3de87-proc\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.924274 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.924273 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c58059a7-6967-4254-b7e7-ca566eb3de87-podres\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.924584 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.924295 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c58059a7-6967-4254-b7e7-ca566eb3de87-lib-modules\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.924584 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.924444 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c58059a7-6967-4254-b7e7-ca566eb3de87-podres\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.924584 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.924459 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c58059a7-6967-4254-b7e7-ca566eb3de87-lib-modules\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:33.932744 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:33.932704 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdpkd\" (UniqueName: \"kubernetes.io/projected/c58059a7-6967-4254-b7e7-ca566eb3de87-kube-api-access-zdpkd\") pod \"perf-node-gather-daemonset-lm99m\" (UID: \"c58059a7-6967-4254-b7e7-ca566eb3de87\") " pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:34.020626 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:34.020579 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:34.317630 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:34.317596 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m"] Sep 19 06:22:34.321246 ip-10-0-142-57 kubenswrapper[2584]: W0919 06:22:34.321211 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc58059a7_6967_4254_b7e7_ca566eb3de87.slice/crio-e97c0ca0d0632ce0523a8671b8dbe281d0fd2d8dbc3e4541fa882f2d05239d2b WatchSource:0}: Error finding container e97c0ca0d0632ce0523a8671b8dbe281d0fd2d8dbc3e4541fa882f2d05239d2b: Status 404 returned error can't find the container with id e97c0ca0d0632ce0523a8671b8dbe281d0fd2d8dbc3e4541fa882f2d05239d2b Sep 19 06:22:34.558930 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:34.558888 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" event={"ID":"c58059a7-6967-4254-b7e7-ca566eb3de87","Type":"ContainerStarted","Data":"eef607a8326d4f9abbf285ad879e87e62ffd64602e2cef8d63331ca59ae8cecf"} Sep 19 06:22:34.559388 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:34.558958 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" event={"ID":"c58059a7-6967-4254-b7e7-ca566eb3de87","Type":"ContainerStarted","Data":"e97c0ca0d0632ce0523a8671b8dbe281d0fd2d8dbc3e4541fa882f2d05239d2b"} Sep 19 06:22:34.559388 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:34.559253 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:34.575318 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:34.575259 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" podStartSLOduration=1.575242778 podStartE2EDuration="1.575242778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 06:22:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 06:22:34.573249943 +0000 UTC m=+2482.566477489" watchObservedRunningTime="2026-09-19 06:22:34.575242778 +0000 UTC m=+2482.568470310" Sep 19 06:22:35.501602 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:35.501528 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lswnv_fe317156-45ab-43ef-b710-c09ac2f3debe/dns/0.log" Sep 19 06:22:35.520809 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:35.520785 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lswnv_fe317156-45ab-43ef-b710-c09ac2f3debe/kube-rbac-proxy/0.log" Sep 19 06:22:35.572315 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:35.572285 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mmcd6_11de77e7-d493-4516-be47-51e244b1c3fe/dns-node-resolver/0.log" Sep 19 06:22:36.146411 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:36.146354 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5fm2w_c30c631d-63ed-410c-8d71-a0f4384bae54/node-ca/0.log" Sep 19 06:22:37.615241 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:37.615210 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b984b98c7-r7tgg_6e631cb4-6f0e-4c0f-9350-8aaff40a84f9/kube-auth-proxy/0.log" Sep 19 06:22:37.678364 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:37.678329 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-spp4h_2c4703ef-5647-4533-a88f-6443cace6aec/payload-pre-processing/0.log" Sep 19 06:22:37.712445 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:37.712409 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dmnts_e3c2b6d2-b60c-4dde-906b-7301fea6c7f5/payload-processing/0.log" Sep 19 06:22:38.695566 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:38.695535 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ct6zn_4b0a35bb-9988-4069-8c94-38de7dff23cb/serve-healthcheck-canary/0.log" Sep 19 06:22:39.235076 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:39.235020 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nz8gq_3acd80a4-9d7b-4f80-98e7-68d5429e8bc3/kube-rbac-proxy/0.log" Sep 19 06:22:39.253625 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:39.253597 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nz8gq_3acd80a4-9d7b-4f80-98e7-68d5429e8bc3/exporter/0.log" Sep 19 06:22:39.272814 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:39.272785 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nz8gq_3acd80a4-9d7b-4f80-98e7-68d5429e8bc3/extractor/0.log" Sep 19 06:22:40.577586 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:40.577251 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5n4fs/perf-node-gather-daemonset-lm99m" Sep 19 06:22:41.999598 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:41.999557 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-687fc9b78d-48kkf_97a3601f-1d00-4289-9c7f-d759b0d552fb/manager/0.log" Sep 19 06:22:42.183943 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:42.183908 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fd6794c5b-ht29j_732f2a56-951f-4a7b-a3b7-2be38ff1c869/manager/0.log" Sep 19 06:22:42.203472 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:42.203436 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fd6794c5b-ht29j_732f2a56-951f-4a7b-a3b7-2be38ff1c869/copy-manifests/0.log" Sep 19 06:22:43.780482 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:43.780361 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58cf5d9b8c-z4jcs_34167abd-02cc-4054-9cad-62f55fe837a3/manager/0.log" Sep 19 06:22:43.805915 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:43.805880 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wh4xh_321e0dd2-f379-4880-a307-476a65bb39e5/openshift-lws-operator/0.log" Sep 19 06:22:44.629878 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:44.629845 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b984b98c7-r7tgg_6e631cb4-6f0e-4c0f-9350-8aaff40a84f9/kube-auth-proxy/0.log" Sep 19 06:22:44.678388 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:44.678352 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-spp4h_2c4703ef-5647-4533-a88f-6443cace6aec/payload-pre-processing/0.log" Sep 19 06:22:44.710741 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:44.710718 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dmnts_e3c2b6d2-b60c-4dde-906b-7301fea6c7f5/payload-processing/0.log" Sep 19 06:22:46.677151 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:46.677114 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b984b98c7-r7tgg_6e631cb4-6f0e-4c0f-9350-8aaff40a84f9/kube-auth-proxy/0.log" Sep 19 06:22:46.729514 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:46.729477 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-spp4h_2c4703ef-5647-4533-a88f-6443cace6aec/payload-pre-processing/0.log" Sep 19 06:22:46.756739 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:46.756705 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-dmnts_e3c2b6d2-b60c-4dde-906b-7301fea6c7f5/payload-processing/0.log" Sep 19 06:22:50.316524 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:50.316478 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7lg7k_ea8b7360-aead-4ea7-858c-047ce3c341b7/kube-multus/0.log" Sep 19 06:22:50.478912 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:50.478882 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gss8z_653a14cf-5131-4a36-8c13-35f8ab9d499a/kube-multus-additional-cni-plugins/0.log" Sep 19 06:22:50.497779 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:50.497744 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gss8z_653a14cf-5131-4a36-8c13-35f8ab9d499a/egress-router-binary-copy/0.log" Sep 19 06:22:50.516353 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:50.516328 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gss8z_653a14cf-5131-4a36-8c13-35f8ab9d499a/cni-plugins/0.log" Sep 19 06:22:50.535539 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:50.535510 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gss8z_653a14cf-5131-4a36-8c13-35f8ab9d499a/bond-cni-plugin/0.log" Sep 19 06:22:50.559895 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:50.559865 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gss8z_653a14cf-5131-4a36-8c13-35f8ab9d499a/routeoverride-cni/0.log" Sep 19 06:22:50.580080 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:50.580061 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gss8z_653a14cf-5131-4a36-8c13-35f8ab9d499a/whereabouts-cni-bincopy/0.log" Sep 19 06:22:50.601201 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:50.601174 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gss8z_653a14cf-5131-4a36-8c13-35f8ab9d499a/whereabouts-cni/0.log" Sep 19 06:22:50.854863 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:50.854782 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8947n_ff709368-cadb-49c8-8bf1-eb0c3938922c/network-metrics-daemon/0.log" Sep 19 06:22:50.870530 ip-10-0-142-57 kubenswrapper[2584]: I0919 06:22:50.870501 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8947n_ff709368-cadb-49c8-8bf1-eb0c3938922c/kube-rbac-proxy/0.log"