Sep 15 07:16:24.620735 ip-10-0-143-175 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 15 07:16:24.620746 ip-10-0-143-175 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 15 07:16:24.620755 ip-10-0-143-175 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 15 07:16:24.621466 ip-10-0-143-175 systemd[1]: Failed to start Kubernetes Kubelet. Sep 15 07:16:34.770158 ip-10-0-143-175 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 15 07:16:34.770176 ip-10-0-143-175 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 4b4855464d554a54963340cfcf97f217 -- Sep 15 07:18:37.495637 ip-10-0-143-175 systemd[1]: Starting Kubernetes Kubelet... Sep 15 07:18:37.843435 ip-10-0-143-175 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 07:18:37.843435 ip-10-0-143-175 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 15 07:18:37.843435 ip-10-0-143-175 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 07:18:37.843435 ip-10-0-143-175 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 15 07:18:37.843435 ip-10-0-143-175 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 07:18:37.844449 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.844340 2552 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 15 07:18:37.848593 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848574 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 07:18:37.848593 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848590 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 07:18:37.848593 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848596 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 15 07:18:37.848593 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848600 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848605 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848611 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848615 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848619 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848623 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848626 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848630 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848643 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848647 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848652 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848656 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848660 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848666 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848671 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848675 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848679 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848682 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848686 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848690 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 07:18:37.848838 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848694 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848698 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848701 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848705 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848709 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848713 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848717 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848722 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848727 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848732 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848736 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848740 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848745 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848750 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848757 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848761 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848766 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848771 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848776 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848780 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 07:18:37.849676 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848784 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848789 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848793 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848797 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848801 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848808 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848813 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848818 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848823 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848828 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848832 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848837 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848841 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848846 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848850 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848854 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848859 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848863 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848868 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 07:18:37.850535 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848872 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848880 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848886 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848891 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848895 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848900 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848905 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848911 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848915 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848920 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848924 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848928 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848933 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848937 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848942 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848946 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848951 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848955 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848959 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 07:18:37.851324 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848963 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848968 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848972 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848977 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848981 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848986 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848990 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848994 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.848998 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.849002 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.849007 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.849011 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.849015 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.849019 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.849023 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.849028 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.849033 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.849037 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850185 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 07:18:37.852211 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850196 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850202 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850206 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850211 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850215 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850219 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850224 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850229 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850234 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850239 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850243 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850247 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850251 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850256 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850260 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850264 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850268 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850273 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850277 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850284 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 07:18:37.853029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850290 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850295 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850301 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850305 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850310 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850315 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850319 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850323 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850328 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850332 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850337 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850341 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850368 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850378 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850383 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850387 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850392 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850396 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850401 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 07:18:37.853702 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850404 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850408 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850413 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850417 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850421 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850426 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850430 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850434 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850438 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850443 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850447 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850452 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850456 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850460 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850465 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850469 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850473 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850477 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850482 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850486 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 07:18:37.854238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850490 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850496 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850502 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850507 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850513 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850518 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850523 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850527 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850531 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850535 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850539 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850543 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850547 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850551 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850555 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850559 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850563 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850567 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850574 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 07:18:37.854858 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850579 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850583 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850588 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850592 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850596 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850601 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850605 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850609 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850613 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850618 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850622 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850626 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850630 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850634 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850638 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850642 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850646 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850650 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850656 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.850661 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 07:18:37.855552 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851236 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851244 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851248 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851253 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851257 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851262 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851266 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851271 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851275 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851280 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851284 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851288 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851292 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851297 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851301 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851306 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851310 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851315 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851319 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851323 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 07:18:37.856072 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851327 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851331 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851335 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851342 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851365 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851371 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851376 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851380 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851385 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851391 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851398 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851403 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851409 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851414 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851418 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851423 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851427 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851431 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851435 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 07:18:37.856669 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851441 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851447 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851452 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851456 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851463 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851469 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851474 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851478 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851483 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851487 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851492 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851496 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851501 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851505 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851510 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851514 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851518 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851523 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851527 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851532 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 07:18:37.857124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851536 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851540 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851544 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851551 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851556 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851561 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851565 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851570 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851575 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851579 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851583 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851587 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851591 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851596 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851599 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851604 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851608 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851612 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851616 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851620 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 07:18:37.857640 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851625 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851629 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851633 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851637 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851641 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851645 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851649 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851653 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851658 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851663 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851668 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851672 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851676 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851681 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851685 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851690 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851694 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851699 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851703 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.851708 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 07:18:37.858158 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851828 2552 flags.go:64] FLAG: --address="0.0.0.0" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851885 2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851896 2552 flags.go:64] FLAG: --anonymous-auth="true" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851903 2552 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851910 2552 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851915 2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851921 2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851928 2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851933 2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851938 2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851944 2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851949 2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851954 2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851959 2552 flags.go:64] FLAG: --cgroup-root="" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851963 2552 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851968 2552 flags.go:64] FLAG: --client-ca-file="" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851972 2552 flags.go:64] FLAG: --cloud-provider="external" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851977 2552 flags.go:64] FLAG: --cluster-dns="[]" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851984 2552 flags.go:64] FLAG: --cluster-domain="" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851989 2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.851994 2552 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852001 2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852007 2552 flags.go:64] FLAG: --container-log-max-files="5" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852012 2552 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 15 07:18:37.858695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852017 2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852022 2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852028 2552 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852035 2552 flags.go:64] FLAG: --contention-profiling="false" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852039 2552 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852044 2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852050 2552 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852055 2552 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852061 2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852067 2552 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852071 2552 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852076 2552 flags.go:64] FLAG: --enable-load-reader="false" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852081 2552 flags.go:64] FLAG: --enable-server="true" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852085 2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852092 2552 flags.go:64] FLAG: --event-burst="100" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852097 2552 flags.go:64] FLAG: --event-qps="50" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852103 2552 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852108 2552 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852113 2552 flags.go:64] FLAG: --eviction-hard="" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852119 2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852123 2552 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852129 2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852134 2552 flags.go:64] FLAG: --eviction-soft="" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852138 2552 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852143 2552 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 15 07:18:37.859263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852148 2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852153 2552 flags.go:64] FLAG: --experimental-mounter-path="" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852158 2552 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852162 2552 flags.go:64] FLAG: --fail-swap-on="true" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852168 2552 flags.go:64] FLAG: --feature-gates="" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852175 2552 flags.go:64] FLAG: --file-check-frequency="20s" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852180 2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852185 2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852190 2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852195 2552 flags.go:64] FLAG: --healthz-port="10248" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852201 2552 flags.go:64] FLAG: --help="false" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852206 2552 flags.go:64] FLAG: --hostname-override="ip-10-0-143-175.ec2.internal" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852212 2552 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852216 2552 flags.go:64] FLAG: --http-check-frequency="20s" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852221 2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852227 2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852233 2552 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852237 2552 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852243 2552 flags.go:64] FLAG: --image-service-endpoint="" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852247 2552 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852252 2552 flags.go:64] FLAG: --kube-api-burst="100" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852257 2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852262 2552 flags.go:64] FLAG: --kube-api-qps="50" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852267 2552 flags.go:64] FLAG: --kube-reserved="" Sep 15 07:18:37.859957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852271 2552 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852276 2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852281 2552 flags.go:64] FLAG: --kubelet-cgroups="" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852285 2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852290 2552 flags.go:64] FLAG: --lock-file="" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852295 2552 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852299 2552 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852304 2552 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852313 2552 flags.go:64] FLAG: --log-json-split-stream="false" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852317 2552 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852322 2552 flags.go:64] FLAG: --log-text-split-stream="false" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852327 2552 flags.go:64] FLAG: --logging-format="text" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852333 2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852339 2552 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852343 2552 flags.go:64] FLAG: --manifest-url="" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852366 2552 flags.go:64] FLAG: --manifest-url-header="" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852373 2552 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852378 2552 flags.go:64] FLAG: --max-open-files="1000000" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852385 2552 flags.go:64] FLAG: --max-pods="110" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852390 2552 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852395 2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852399 2552 flags.go:64] FLAG: --memory-manager-policy="None" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852404 2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852410 2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852415 2552 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 15 07:18:37.860565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852420 2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852431 2552 flags.go:64] FLAG: --node-status-max-images="50" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852436 2552 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852441 2552 flags.go:64] FLAG: --oom-score-adj="-999" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852447 2552 flags.go:64] FLAG: --pod-cidr="" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852451 2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852460 2552 flags.go:64] FLAG: --pod-manifest-path="" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852464 2552 flags.go:64] FLAG: --pod-max-pids="-1" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852469 2552 flags.go:64] FLAG: --pods-per-core="0" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852474 2552 flags.go:64] FLAG: --port="10250" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852479 2552 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852484 2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-087e8bd18d366ed23" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852489 2552 flags.go:64] FLAG: --qos-reserved="" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852494 2552 flags.go:64] FLAG: --read-only-port="10255" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852499 2552 flags.go:64] FLAG: --register-node="true" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852503 2552 flags.go:64] FLAG: --register-with-taints="" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852509 2552 flags.go:64] FLAG: --registry-burst="10" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852514 2552 flags.go:64] FLAG: --registry-qps="5" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852519 2552 flags.go:64] FLAG: --reserved-cpus="" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852524 2552 flags.go:64] FLAG: --reserved-memory="" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852532 2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852537 2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852542 2552 flags.go:64] FLAG: --rotate-certificates="false" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852546 2552 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852551 2552 flags.go:64] FLAG: --runonce="false" Sep 15 07:18:37.861171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852557 2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852562 2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852567 2552 flags.go:64] FLAG: --seccomp-default="false" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852571 2552 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852576 2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852581 2552 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852586 2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852592 2552 flags.go:64] FLAG: --storage-driver-password="root" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852597 2552 flags.go:64] FLAG: --storage-driver-secure="false" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852602 2552 flags.go:64] FLAG: --storage-driver-table="stats" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852606 2552 flags.go:64] FLAG: --storage-driver-user="root" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852611 2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852616 2552 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852621 2552 flags.go:64] FLAG: --system-cgroups="" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852625 2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852634 2552 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852638 2552 flags.go:64] FLAG: --tls-cert-file="" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852643 2552 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852650 2552 flags.go:64] FLAG: --tls-min-version="" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852654 2552 flags.go:64] FLAG: --tls-private-key-file="" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852659 2552 flags.go:64] FLAG: --topology-manager-policy="none" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852663 2552 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852668 2552 flags.go:64] FLAG: --topology-manager-scope="container" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852673 2552 flags.go:64] FLAG: --v="2" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852679 2552 flags.go:64] FLAG: --version="false" Sep 15 07:18:37.861923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852685 2552 flags.go:64] FLAG: --vmodule="" Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852691 2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.852698 2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852899 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852905 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852910 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852914 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852919 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852923 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852927 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852931 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852935 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852939 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852943 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852948 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852953 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852957 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852961 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852966 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852970 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 07:18:37.862827 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852974 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852978 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852983 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852987 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852991 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.852995 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853000 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853004 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853008 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853012 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853016 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853020 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853025 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853029 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853035 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853039 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853042 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853047 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853051 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853055 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 07:18:37.863314 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853059 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853063 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853067 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853072 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853077 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853081 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853086 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853091 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853095 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853099 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853104 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853108 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853112 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853116 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853120 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853126 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853131 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853135 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853141 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 07:18:37.863875 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853148 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853154 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853158 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853164 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853170 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853176 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853180 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853187 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853192 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853196 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853200 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853205 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853209 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853213 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853217 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853221 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853225 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853229 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853234 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 07:18:37.864331 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853238 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853244 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853250 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853256 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853260 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853264 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853269 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853274 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853278 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853284 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853288 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853292 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853296 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853300 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853304 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853308 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853313 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853317 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853321 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 07:18:37.864818 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853325 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853331 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853335 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853340 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.853344 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.853873 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.860806 2552 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.860820 2552 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.861545 2552 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.861553 2552 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862848 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862862 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862865 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862868 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862872 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 07:18:37.865304 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862875 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862878 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862882 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862885 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862888 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862892 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862895 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862898 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862900 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862904 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862907 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862910 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862913 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862915 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862918 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862921 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862924 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862927 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862929 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862932 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 07:18:37.865699 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862934 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862937 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862940 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862943 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862945 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862955 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862959 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862962 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862966 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862970 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862973 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862976 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862979 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862982 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862985 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862988 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862991 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862995 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.862998 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863001 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 07:18:37.866175 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863004 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863007 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863010 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863012 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863014 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863018 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863021 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863023 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863026 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863029 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863032 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863035 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863037 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863040 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863042 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863045 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863047 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863050 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863052 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863055 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 07:18:37.866687 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863059 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863063 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863067 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863069 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863072 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863075 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863078 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863080 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863083 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863085 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863088 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863090 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863093 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863095 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863098 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863100 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863103 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863105 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863109 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863112 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 07:18:37.867163 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863115 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863117 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863120 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863122 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863126 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863130 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863132 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863135 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863138 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863141 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863144 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863147 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863150 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863153 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 07:18:37.867723 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.863159 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863282 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863287 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863290 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863293 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863296 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863298 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863301 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863303 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863306 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863308 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863311 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863313 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863316 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863320 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863325 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863328 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863331 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863334 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863336 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 07:18:37.868080 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863339 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863342 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863344 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863347 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863364 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863367 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863369 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863372 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863375 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863379 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863382 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863386 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863389 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863392 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863395 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863397 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863400 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863403 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863406 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 07:18:37.868587 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863408 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863411 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863413 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863416 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863418 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863421 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863423 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863426 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863429 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863431 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863434 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863437 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863439 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863442 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863445 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863448 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863450 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863453 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863455 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863458 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 07:18:37.869036 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863461 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863463 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863466 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863469 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863472 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863474 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863477 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863480 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863482 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863485 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863488 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863490 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863493 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863495 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863498 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863500 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863503 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863506 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863509 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863513 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 07:18:37.869530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863516 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863519 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863522 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863524 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863527 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863530 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863533 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863536 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863538 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863541 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863544 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863546 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863548 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863551 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863553 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863556 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863559 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863562 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863564 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863567 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 07:18:37.870004 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:37.863569 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 07:18:37.870502 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.863574 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 07:18:37.870502 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.863673 2552 server.go:962] "Client rotation is on, will bootstrap in background" Sep 15 07:18:37.870502 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.867172 2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 15 07:18:37.870502 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.867970 2552 server.go:1019] "Starting client certificate rotation" Sep 15 07:18:37.870502 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.868075 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 15 07:18:37.870502 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.868719 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 15 07:18:37.870502 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.870280 2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 15 07:18:37.875655 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.875630 2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 15 07:18:37.888840 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.888824 2552 log.go:25] "Validated CRI v1 runtime API" Sep 15 07:18:37.891428 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.891415 2552 log.go:25] "Validated CRI v1 image API" Sep 15 07:18:37.892272 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.892254 2552 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 15 07:18:37.894861 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.894838 2552 fs.go:135] Filesystem UUIDs: map[1936787a-1720-4651-a250-6f48e33a7703:/dev/nvme0n1p3 71ed2f58-5a47-4746-91be-c1e5bc2d3c96:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Sep 15 07:18:37.894948 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.894857 2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 15 07:18:37.895259 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.895244 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 15 07:18:37.899908 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.899794 2552 manager.go:216] Machine: {Timestamp:2026-09-15 07:18:37.898747689 +0000 UTC m=+0.299424258 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099625 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fe7630af866f17e4e08e69eaa8232 SystemUUID:ec2fe763-0af8-66f1-7e4e-08e69eaa8232 BootID:4b485546-4d55-4a54-9633-40cfcf97f217 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fc:b6:8b:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fc:b6:8b:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:bf:44:af:5c:d9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 15 07:18:37.899908 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.899902 2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 15 07:18:37.900015 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.899974 2552 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 15 07:18:37.901661 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.901629 2552 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 15 07:18:37.901823 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.901671 2552 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-175.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 15 07:18:37.902445 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.902435 2552 topology_manager.go:138] "Creating topology manager with none policy" Sep 15 07:18:37.902484 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.902447 2552 container_manager_linux.go:309] "Creating device plugin manager" Sep 15 07:18:37.902484 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.902454 2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 15 07:18:37.903019 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.903009 2552 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 15 07:18:37.903845 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.903835 2552 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 15 07:18:37.906594 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.906583 2552 state_mem.go:36] "Initialized new in-memory state store" Sep 15 07:18:37.906752 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.906743 2552 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 15 07:18:37.906802 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.906795 2552 kubelet.go:486] "Attempting to sync node with API server" Sep 15 07:18:37.906831 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.906815 2552 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 15 07:18:37.906864 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.906831 2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 15 07:18:37.906864 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.906844 2552 kubelet.go:398] "Adding apiserver pod source" Sep 15 07:18:37.906864 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.906855 2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 15 07:18:37.907337 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.907318 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 15 07:18:37.907434 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.907372 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 15 07:18:37.911588 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.911570 2552 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 15 07:18:37.913844 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.913827 2552 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 15 07:18:37.913918 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.913857 2552 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 15 07:18:37.915249 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915236 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 15 07:18:37.915320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915253 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 15 07:18:37.915320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915260 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 15 07:18:37.915320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915266 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 15 07:18:37.915320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915274 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 15 07:18:37.915320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915280 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 15 07:18:37.915320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915293 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 15 07:18:37.915320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915298 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 15 07:18:37.915320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915305 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 15 07:18:37.915320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915310 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 15 07:18:37.915320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915318 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 15 07:18:37.915320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915323 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 15 07:18:37.915886 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915875 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 15 07:18:37.915886 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.915885 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 15 07:18:37.916146 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:37.916122 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 15 07:18:37.916195 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:37.916122 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-175.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 15 07:18:37.917489 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.917470 2552 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-175.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 15 07:18:37.919363 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.919339 2552 server.go:1268] "Started kubelet" Sep 15 07:18:37.919427 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.919408 2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 15 07:18:37.919518 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.919479 2552 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 15 07:18:37.919558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.919535 2552 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 15 07:18:37.920631 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.920613 2552 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 15 07:18:37.920998 ip-10-0-143-175 systemd[1]: Started Kubernetes Kubelet. Sep 15 07:18:37.921178 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.921045 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dph7c" Sep 15 07:18:37.921821 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.921807 2552 server.go:310] "Adding debug handlers to kubelet server" Sep 15 07:18:37.923465 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:37.923448 2552 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 15 07:18:37.925478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.925465 2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 15 07:18:37.925533 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.925486 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 15 07:18:37.925619 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:37.925584 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:37.925976 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.925955 2552 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 15 07:18:37.926095 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.926076 2552 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 15 07:18:37.926168 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.926098 2552 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 15 07:18:37.926226 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.926212 2552 reconstruct.go:97] "Volume reconstruction finished" Sep 15 07:18:37.926277 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.926231 2552 reconciler.go:29] "Reconciler: start to sync state" Sep 15 07:18:37.926688 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.926673 2552 factory.go:55] Registering systemd factory Sep 15 07:18:37.926688 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.926688 2552 factory.go:223] Registration of the systemd container factory successfully Sep 15 07:18:37.926899 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.926880 2552 factory.go:153] Registering CRI-O factory Sep 15 07:18:37.926899 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.926899 2552 factory.go:223] Registration of the crio container factory successfully Sep 15 07:18:37.926990 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.926959 2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 15 07:18:37.926990 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.926980 2552 factory.go:103] Registering Raw factory Sep 15 07:18:37.927058 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.927006 2552 manager.go:1196] Started watching for new ooms in manager Sep 15 07:18:37.927647 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.927632 2552 manager.go:349] Starting recovery of all containers Sep 15 07:18:37.928817 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.928795 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dph7c" Sep 15 07:18:37.931134 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:37.931109 2552 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-175.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 15 07:18:37.931994 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:37.931199 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-175.ec2.internal.18d56d676c845f12 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-175.ec2.internal,UID:ip-10-0-143-175.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-175.ec2.internal,},FirstTimestamp:2026-09-15 07:18:37.919313682 +0000 UTC m=+0.319990250,LastTimestamp:2026-09-15 07:18:37.919313682 +0000 UTC m=+0.319990250,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-175.ec2.internal,}" Sep 15 07:18:37.932100 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:37.932079 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 15 07:18:37.941232 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.941114 2552 manager.go:354] Recovery completed Sep 15 07:18:37.946313 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.946301 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:18:37.951814 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.951797 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:18:37.951913 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.951829 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:18:37.951913 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.951845 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:18:37.952475 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.952462 2552 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 15 07:18:37.952475 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.952475 2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 15 07:18:37.952600 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.952491 2552 state_mem.go:36] "Initialized new in-memory state store" Sep 15 07:18:37.952600 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.952509 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:18:37.953259 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.953243 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:18:37.953320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.953269 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:18:37.953320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.953283 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:18:37.953719 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.953706 2552 policy_none.go:49] "None policy: Start" Sep 15 07:18:37.953755 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.953724 2552 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 15 07:18:37.953755 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.953734 2552 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 15 07:18:37.954711 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.954700 2552 policy_none.go:47] "Start" Sep 15 07:18:37.978925 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.978911 2552 manager.go:337] "Starting Device Plugin manager" Sep 15 07:18:37.987754 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:37.978943 2552 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 15 07:18:37.987754 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.978953 2552 server.go:87] "Starting device plugin registration server" Sep 15 07:18:37.987754 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.979244 2552 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 15 07:18:37.987754 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.979258 2552 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 15 07:18:37.987754 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.979367 2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 15 07:18:37.987754 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.979452 2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 15 07:18:37.987754 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:37.979464 2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 15 07:18:37.987754 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:37.980096 2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 15 07:18:37.987754 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:37.980133 2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:38.047103 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.047075 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 15 07:18:38.048289 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.048270 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 15 07:18:38.048395 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.048321 2552 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 15 07:18:38.048395 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.048346 2552 kubelet.go:2448] "Starting kubelet main sync loop" Sep 15 07:18:38.048500 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:38.048462 2552 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 15 07:18:38.050730 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.050714 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 07:18:38.079597 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.079582 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:18:38.080484 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.080467 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:18:38.080561 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.080499 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:18:38.080561 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.080513 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:18:38.080561 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.080540 2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.086280 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.086264 2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.086376 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:38.086291 2552 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-175.ec2.internal\": node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:38.112434 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:38.112380 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:38.148938 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.148919 2552 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal"] Sep 15 07:18:38.149025 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.148976 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:18:38.150291 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.150275 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:18:38.150379 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.150303 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:18:38.150379 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.150312 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:18:38.151568 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.151556 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:18:38.151730 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.151717 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.151814 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.151757 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:18:38.152244 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.152232 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:18:38.152290 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.152257 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:18:38.152290 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.152266 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:18:38.152372 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.152236 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:18:38.152372 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.152328 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:18:38.152372 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.152338 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:18:38.153373 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.153346 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.153421 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.153398 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 07:18:38.154065 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.154050 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientMemory" Sep 15 07:18:38.154150 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.154079 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 07:18:38.154150 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.154093 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeHasSufficientPID" Sep 15 07:18:38.176985 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:38.176969 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-175.ec2.internal\" not found" node="ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.181404 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:38.181389 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-175.ec2.internal\" not found" node="ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.213399 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:38.213384 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:38.227580 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.227564 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/438588d6d557fa6d076676dc45773c05-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal\" (UID: \"438588d6d557fa6d076676dc45773c05\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.227670 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.227588 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/438588d6d557fa6d076676dc45773c05-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal\" (UID: \"438588d6d557fa6d076676dc45773c05\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.227670 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.227606 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff4bf43fd141c579255e728a935942fa-config\") pod \"kube-apiserver-proxy-ip-10-0-143-175.ec2.internal\" (UID: \"ff4bf43fd141c579255e728a935942fa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.313842 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:38.313816 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:38.328185 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.328163 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/438588d6d557fa6d076676dc45773c05-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal\" (UID: \"438588d6d557fa6d076676dc45773c05\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.328257 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.328167 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/438588d6d557fa6d076676dc45773c05-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal\" (UID: \"438588d6d557fa6d076676dc45773c05\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.328257 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.328219 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/438588d6d557fa6d076676dc45773c05-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal\" (UID: \"438588d6d557fa6d076676dc45773c05\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.328257 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.328238 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff4bf43fd141c579255e728a935942fa-config\") pod \"kube-apiserver-proxy-ip-10-0-143-175.ec2.internal\" (UID: \"ff4bf43fd141c579255e728a935942fa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.328257 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.328256 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ff4bf43fd141c579255e728a935942fa-config\") pod \"kube-apiserver-proxy-ip-10-0-143-175.ec2.internal\" (UID: \"ff4bf43fd141c579255e728a935942fa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.328414 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.328277 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/438588d6d557fa6d076676dc45773c05-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal\" (UID: \"438588d6d557fa6d076676dc45773c05\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.414613 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:38.414545 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:38.478985 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.478947 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.483732 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:38.483716 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" Sep 15 07:18:38.515426 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:38.515397 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:38.616669 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:38.616622 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:40.136590 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:40.136562 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:40.237122 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:40.237085 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:40.288516 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:40.288491 2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 15 07:18:40.288907 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:40.288628 2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 15 07:18:40.338070 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:40.338043 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:40.352718 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:40.352666 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 07:13:39 +0000 UTC" deadline="2028-02-27 13:12:14.339023696 +0000 UTC" Sep 15 07:18:40.352718 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:40.352687 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12725h53m33.986339232s" Sep 15 07:18:40.439082 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:40.439058 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:40.537806 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:40.537776 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod438588d6d557fa6d076676dc45773c05.slice/crio-92fb934314879b68c2f9a34f643b1f29032f5c8673155cd268d449a5442856ea WatchSource:0}: Error finding container 92fb934314879b68c2f9a34f643b1f29032f5c8673155cd268d449a5442856ea: Status 404 returned error can't find the container with id 92fb934314879b68c2f9a34f643b1f29032f5c8673155cd268d449a5442856ea Sep 15 07:18:40.538603 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:40.538579 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4bf43fd141c579255e728a935942fa.slice/crio-0199abbeff2780baffc827cd1adae33cc281c32133937616d5ac04b4d6fe4660 WatchSource:0}: Error finding container 0199abbeff2780baffc827cd1adae33cc281c32133937616d5ac04b4d6fe4660: Status 404 returned error can't find the container with id 0199abbeff2780baffc827cd1adae33cc281c32133937616d5ac04b4d6fe4660 Sep 15 07:18:40.539644 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:40.539621 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:40.543039 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:40.543018 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 07:18:40.581930 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:40.581907 2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 07:18:40.639782 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:40.639720 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:40.740252 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:40.740213 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:40.841052 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:40.841019 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-175.ec2.internal\" not found" Sep 15 07:18:40.878526 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:40.878498 2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 07:18:40.931831 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:40.931771 2552 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 07:18:40.945999 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:40.945964 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" Sep 15 07:18:40.956040 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:40.956025 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 15 07:18:40.956973 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:40.956962 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" Sep 15 07:18:40.962890 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:40.962863 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 15 07:18:41.327681 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.327594 2552 apiserver.go:52] "Watching apiserver" Sep 15 07:18:41.334307 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.334280 2552 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 15 07:18:41.334804 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.334777 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-njnqn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal","openshift-network-operator/iptables-alerter-r4mbg","openshift-ovn-kubernetes/ovnkube-node-xkh69","kube-system/global-pull-secret-syncer-h8b5m","kube-system/konnectivity-agent-9mt82","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv","openshift-dns/node-resolver-xbtjp","openshift-multus/multus-additional-cni-plugins-r2hmp","openshift-multus/multus-djmmr","openshift-multus/network-metrics-daemon-n9rxl","openshift-network-diagnostics/network-check-target-vbjvp","kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal","openshift-cluster-node-tuning-operator/tuned-dfrxr"] Sep 15 07:18:41.336454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.336434 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.337788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.337762 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:41.337911 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.337881 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:18:41.338827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.338802 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 15 07:18:41.338958 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.338853 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 15 07:18:41.338958 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.338888 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k42fj\"" Sep 15 07:18:41.339155 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.339128 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r4mbg" Sep 15 07:18:41.339862 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.339841 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 15 07:18:41.340576 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.340558 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.341360 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.341265 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 15 07:18:41.341360 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.341265 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 15 07:18:41.341360 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.341271 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 15 07:18:41.341563 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.341530 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tk4jp\"" Sep 15 07:18:41.342465 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.342447 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 15 07:18:41.342465 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.342457 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:41.342607 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.342516 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:18:41.343050 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.343032 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 15 07:18:41.343365 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.343119 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 15 07:18:41.343365 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.343247 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 15 07:18:41.343547 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.343528 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 15 07:18:41.343603 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.343550 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 15 07:18:41.343807 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.343788 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2fvnh\"" Sep 15 07:18:41.344068 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.344050 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9mt82" Sep 15 07:18:41.346259 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.346238 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 15 07:18:41.347488 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.347468 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-llbfl\"" Sep 15 07:18:41.347740 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.347725 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 15 07:18:41.348193 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.348007 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 15 07:18:41.348942 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.348330 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.350746 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.350043 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.350915 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.350898 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 15 07:18:41.351019 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.350966 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 15 07:18:41.351596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.351240 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 15 07:18:41.351596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.351334 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-66pt5\"" Sep 15 07:18:41.351596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.351577 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 15 07:18:41.352603 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.352370 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xbtjp" Sep 15 07:18:41.353239 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.353220 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 15 07:18:41.353762 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.353454 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 15 07:18:41.353762 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.353621 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vvrwr\"" Sep 15 07:18:41.353934 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.353820 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 15 07:18:41.353991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.353955 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.354923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.354708 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 15 07:18:41.354923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.354736 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 15 07:18:41.355274 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355247 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-njnqn" Sep 15 07:18:41.355368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355348 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.355431 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355380 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b931b03f-d393-4723-b2fe-bc49c3475e67-system-cni-dir\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.355431 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355410 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b8eab75a-9838-4a5e-b7c9-b46b1d616807-iptables-alerter-script\") pod \"iptables-alerter-r4mbg\" (UID: \"b8eab75a-9838-4a5e-b7c9-b46b1d616807\") " pod="openshift-network-operator/iptables-alerter-r4mbg" Sep 15 07:18:41.355531 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355434 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b8eab75a-9838-4a5e-b7c9-b46b1d616807-host-slash\") pod \"iptables-alerter-r4mbg\" (UID: \"b8eab75a-9838-4a5e-b7c9-b46b1d616807\") " pod="openshift-network-operator/iptables-alerter-r4mbg" Sep 15 07:18:41.355531 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355459 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b1e23db-0eff-418c-8f36-3e5dedc822ba-agent-certs\") pod \"konnectivity-agent-9mt82\" (UID: \"4b1e23db-0eff-418c-8f36-3e5dedc822ba\") " pod="kube-system/konnectivity-agent-9mt82" Sep 15 07:18:41.355531 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355482 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-socket-dir\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.355531 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355507 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-registration-dir\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.355737 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355531 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6f6820e-f789-494a-9a66-fe16e0c2f21a-dbus\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:41.355737 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355553 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:41.355737 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355579 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:41.355737 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355600 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-cni-netd\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.355737 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355623 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/686793e7-d253-4516-9e2a-e53f8f528cf4-env-overrides\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.355737 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355648 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-log-socket\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.355737 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355672 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-run-ovn-kubernetes\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.355737 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355694 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/686793e7-d253-4516-9e2a-e53f8f528cf4-ovnkube-script-lib\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.355737 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355717 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78x67\" (UniqueName: \"kubernetes.io/projected/b931b03f-d393-4723-b2fe-bc49c3475e67-kube-api-access-78x67\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.356169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355746 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb7l9\" (UniqueName: \"kubernetes.io/projected/b8eab75a-9838-4a5e-b7c9-b46b1d616807-kube-api-access-bb7l9\") pod \"iptables-alerter-r4mbg\" (UID: \"b8eab75a-9838-4a5e-b7c9-b46b1d616807\") " pod="openshift-network-operator/iptables-alerter-r4mbg" Sep 15 07:18:41.356169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355803 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-run-openvswitch\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.356169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355851 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/686793e7-d253-4516-9e2a-e53f8f528cf4-ovn-node-metrics-cert\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.356169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355949 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b931b03f-d393-4723-b2fe-bc49c3475e67-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.356169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355982 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dp2bw\"" Sep 15 07:18:41.356169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.355999 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-device-dir\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.356169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356024 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-etc-selinux\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.356169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356060 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6xpc\" (UniqueName: \"kubernetes.io/projected/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-kube-api-access-s6xpc\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.356169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356101 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htkll\" (UniqueName: \"kubernetes.io/projected/f13cc492-490d-478f-b799-6127572cc075-kube-api-access-htkll\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:41.356169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356170 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-systemd-units\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356206 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b931b03f-d393-4723-b2fe-bc49c3475e67-cni-binary-copy\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356223 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rrg7l\"" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356230 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b931b03f-d393-4723-b2fe-bc49c3475e67-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356261 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b931b03f-d393-4723-b2fe-bc49c3475e67-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356308 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-sys-fs\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356314 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356331 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-kubelet\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356390 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-slash\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356427 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/686793e7-d253-4516-9e2a-e53f8f528cf4-ovnkube-config\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356449 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrdnt\" (UniqueName: \"kubernetes.io/projected/686793e7-d253-4516-9e2a-e53f8f528cf4-kube-api-access-rrdnt\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356496 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6f6820e-f789-494a-9a66-fe16e0c2f21a-kubelet-config\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356524 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b931b03f-d393-4723-b2fe-bc49c3475e67-cnibin\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356535 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356550 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356576 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-run-systemd\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.356585 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356599 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b1e23db-0eff-418c-8f36-3e5dedc822ba-konnectivity-ca\") pod \"konnectivity-agent-9mt82\" (UID: \"4b1e23db-0eff-418c-8f36-3e5dedc822ba\") " pod="kube-system/konnectivity-agent-9mt82" Sep 15 07:18:41.356649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356623 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-run-ovn\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.357388 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356668 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-run-netns\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.357388 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356691 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-var-lib-openvswitch\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.357388 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356714 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-etc-openvswitch\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.357388 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356735 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-node-log\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.357388 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.356755 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-cni-bin\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.357388 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.357349 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 15 07:18:41.357644 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.357396 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 15 07:18:41.357644 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.357633 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 15 07:18:41.357769 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.357754 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-g7dn9\"" Sep 15 07:18:41.359348 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.359177 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 15 07:18:41.377414 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.377392 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bl9jx" Sep 15 07:18:41.384592 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.384570 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bl9jx" Sep 15 07:18:41.447311 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.447276 2552 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 15 07:18:41.457318 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457277 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-hostroot\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.457461 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457327 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-var-lib-openvswitch\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.457461 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457353 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-cni-bin\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.457461 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457412 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-cni-bin\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.457461 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457412 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-var-lib-openvswitch\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.457623 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457458 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.457623 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457491 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b931b03f-d393-4723-b2fe-bc49c3475e67-system-cni-dir\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.457623 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457517 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b8eab75a-9838-4a5e-b7c9-b46b1d616807-host-slash\") pod \"iptables-alerter-r4mbg\" (UID: \"b8eab75a-9838-4a5e-b7c9-b46b1d616807\") " pod="openshift-network-operator/iptables-alerter-r4mbg" Sep 15 07:18:41.457623 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457537 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b1e23db-0eff-418c-8f36-3e5dedc822ba-agent-certs\") pod \"konnectivity-agent-9mt82\" (UID: \"4b1e23db-0eff-418c-8f36-3e5dedc822ba\") " pod="kube-system/konnectivity-agent-9mt82" Sep 15 07:18:41.457623 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457563 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-sys\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.457623 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457560 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.457623 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457585 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b931b03f-d393-4723-b2fe-bc49c3475e67-system-cni-dir\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.457623 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457602 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b8eab75a-9838-4a5e-b7c9-b46b1d616807-host-slash\") pod \"iptables-alerter-r4mbg\" (UID: \"b8eab75a-9838-4a5e-b7c9-b46b1d616807\") " pod="openshift-network-operator/iptables-alerter-r4mbg" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457637 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/72e4df05-e7be-4167-b564-1fb5ccf015ce-multus-daemon-config\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457667 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-registration-dir\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457683 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-multus-cni-dir\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457700 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-var-lib-cni-bin\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457736 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxm2l\" (UniqueName: \"kubernetes.io/projected/72e4df05-e7be-4167-b564-1fb5ccf015ce-kube-api-access-jxm2l\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457751 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-registration-dir\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457761 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457798 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-kubernetes\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457829 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-log-socket\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.457841 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457862 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-run-ovn-kubernetes\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.457916 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret podName:c6f6820e-f789-494a-9a66-fe16e0c2f21a nodeName:}" failed. No retries permitted until 2026-09-15 07:18:41.957895231 +0000 UTC m=+2.939057393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret") pod "global-pull-secret-syncer-h8b5m" (UID: "c6f6820e-f789-494a-9a66-fe16e0c2f21a") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457928 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-run-ovn-kubernetes\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457933 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78x67\" (UniqueName: \"kubernetes.io/projected/b931b03f-d393-4723-b2fe-bc49c3475e67-kube-api-access-78x67\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.457959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457959 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb7l9\" (UniqueName: \"kubernetes.io/projected/b8eab75a-9838-4a5e-b7c9-b46b1d616807-kube-api-access-bb7l9\") pod \"iptables-alerter-r4mbg\" (UID: \"b8eab75a-9838-4a5e-b7c9-b46b1d616807\") " pod="openshift-network-operator/iptables-alerter-r4mbg" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458008 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42649c54-356b-4265-9d26-2c42d9b0f265-tmp-dir\") pod \"node-resolver-xbtjp\" (UID: \"42649c54-356b-4265-9d26-2c42d9b0f265\") " pod="openshift-dns/node-resolver-xbtjp" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458044 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72e4df05-e7be-4167-b564-1fb5ccf015ce-cni-binary-copy\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458069 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-run-netns\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458093 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-run-openvswitch\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.457955 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-log-socket\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458120 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b931b03f-d393-4723-b2fe-bc49c3475e67-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458160 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-etc-selinux\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458181 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-run-openvswitch\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458196 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6xpc\" (UniqueName: \"kubernetes.io/projected/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-kube-api-access-s6xpc\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458219 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-lib-modules\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458241 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-etc-selinux\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458299 2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458343 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-cnibin\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458373 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b931b03f-d393-4723-b2fe-bc49c3475e67-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458400 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-run-multus-certs\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458426 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-systemd\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.458772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458448 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-run\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458472 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrdnt\" (UniqueName: \"kubernetes.io/projected/686793e7-d253-4516-9e2a-e53f8f528cf4-kube-api-access-rrdnt\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458496 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458518 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-modprobe-d\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458538 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-var-lib-cni-multus\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458560 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-var-lib-kubelet\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458588 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-run-systemd\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458602 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458614 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-multus-conf-dir\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458663 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-run-ovn\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458683 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-run-systemd\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458725 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-run-ovn\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458730 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-run-netns\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458691 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-run-netns\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458764 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b931b03f-d393-4723-b2fe-bc49c3475e67-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458800 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-etc-openvswitch\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458769 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-etc-openvswitch\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.459579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458836 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-node-log\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458885 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b8eab75a-9838-4a5e-b7c9-b46b1d616807-iptables-alerter-script\") pod \"iptables-alerter-r4mbg\" (UID: \"b8eab75a-9838-4a5e-b7c9-b46b1d616807\") " pod="openshift-network-operator/iptables-alerter-r4mbg" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458899 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-node-log\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458912 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8428492-dcc5-4c33-bcd2-29002680f011-tmp\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458922 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b931b03f-d393-4723-b2fe-bc49c3475e67-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.458939 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfwb4\" (UniqueName: \"kubernetes.io/projected/7e2134f2-9ba8-432a-9ba9-aa8e313a86c6-kube-api-access-kfwb4\") pod \"node-ca-njnqn\" (UID: \"7e2134f2-9ba8-432a-9ba9-aa8e313a86c6\") " pod="openshift-image-registry/node-ca-njnqn" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459018 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-system-cni-dir\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459054 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-socket-dir\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459114 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c8428492-dcc5-4c33-bcd2-29002680f011-etc-tuned\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459139 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtdg2\" (UniqueName: \"kubernetes.io/projected/42649c54-356b-4265-9d26-2c42d9b0f265-kube-api-access-mtdg2\") pod \"node-resolver-xbtjp\" (UID: \"42649c54-356b-4265-9d26-2c42d9b0f265\") " pod="openshift-dns/node-resolver-xbtjp" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459160 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-socket-dir\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459170 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e2134f2-9ba8-432a-9ba9-aa8e313a86c6-serviceca\") pod \"node-ca-njnqn\" (UID: \"7e2134f2-9ba8-432a-9ba9-aa8e313a86c6\") " pod="openshift-image-registry/node-ca-njnqn" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459204 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-run-k8s-cni-cncf-io\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459233 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6f6820e-f789-494a-9a66-fe16e0c2f21a-dbus\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459263 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459321 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c6f6820e-f789-494a-9a66-fe16e0c2f21a-dbus\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459349 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-cni-netd\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.460433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459375 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/686793e7-d253-4516-9e2a-e53f8f528cf4-env-overrides\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459384 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-cni-netd\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459396 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f5nc\" (UniqueName: \"kubernetes.io/projected/c8428492-dcc5-4c33-bcd2-29002680f011-kube-api-access-6f5nc\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.459349 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459413 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/42649c54-356b-4265-9d26-2c42d9b0f265-hosts-file\") pod \"node-resolver-xbtjp\" (UID: \"42649c54-356b-4265-9d26-2c42d9b0f265\") " pod="openshift-dns/node-resolver-xbtjp" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459428 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-multus-socket-dir-parent\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459446 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/686793e7-d253-4516-9e2a-e53f8f528cf4-ovnkube-script-lib\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.459469 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs podName:f13cc492-490d-478f-b799-6127572cc075 nodeName:}" failed. No retries permitted until 2026-09-15 07:18:41.95945135 +0000 UTC m=+2.940613516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs") pod "network-metrics-daemon-n9rxl" (UID: "f13cc492-490d-478f-b799-6127572cc075") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459497 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-sysconfig\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459523 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/686793e7-d253-4516-9e2a-e53f8f528cf4-ovn-node-metrics-cert\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459543 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-device-dir\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459565 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e2134f2-9ba8-432a-9ba9-aa8e313a86c6-host\") pod \"node-ca-njnqn\" (UID: \"7e2134f2-9ba8-432a-9ba9-aa8e313a86c6\") " pod="openshift-image-registry/node-ca-njnqn" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459597 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b8eab75a-9838-4a5e-b7c9-b46b1d616807-iptables-alerter-script\") pod \"iptables-alerter-r4mbg\" (UID: \"b8eab75a-9838-4a5e-b7c9-b46b1d616807\") " pod="openshift-network-operator/iptables-alerter-r4mbg" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459604 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlgn6\" (UniqueName: \"kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6\") pod \"network-check-target-vbjvp\" (UID: \"5ecac077-a8a5-4930-a69c-21402e834160\") " pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459629 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-etc-kubernetes\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459652 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htkll\" (UniqueName: \"kubernetes.io/projected/f13cc492-490d-478f-b799-6127572cc075-kube-api-access-htkll\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:41.461333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459673 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-systemd-units\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459696 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b931b03f-d393-4723-b2fe-bc49c3475e67-cni-binary-copy\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459720 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b931b03f-d393-4723-b2fe-bc49c3475e67-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459743 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-sys-fs\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459767 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-sysctl-conf\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459789 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-var-lib-kubelet\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459802 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/686793e7-d253-4516-9e2a-e53f8f528cf4-env-overrides\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459810 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-host\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459833 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-sysctl-d\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459885 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-kubelet\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459889 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-systemd-units\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459907 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-slash\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459922 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/686793e7-d253-4516-9e2a-e53f8f528cf4-ovnkube-script-lib\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459940 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-device-dir\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459928 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/686793e7-d253-4516-9e2a-e53f8f528cf4-ovnkube-config\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.459994 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6f6820e-f789-494a-9a66-fe16e0c2f21a-kubelet-config\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.460016 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-sys-fs\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.462119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.460027 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b931b03f-d393-4723-b2fe-bc49c3475e67-cnibin\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.462929 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.460053 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b1e23db-0eff-418c-8f36-3e5dedc822ba-konnectivity-ca\") pod \"konnectivity-agent-9mt82\" (UID: \"4b1e23db-0eff-418c-8f36-3e5dedc822ba\") " pod="kube-system/konnectivity-agent-9mt82" Sep 15 07:18:41.462929 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.460093 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-kubelet\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.462929 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.460126 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/686793e7-d253-4516-9e2a-e53f8f528cf4-host-slash\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.462929 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.460270 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b931b03f-d393-4723-b2fe-bc49c3475e67-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.462929 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.460314 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c6f6820e-f789-494a-9a66-fe16e0c2f21a-kubelet-config\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:41.462929 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.460363 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b931b03f-d393-4723-b2fe-bc49c3475e67-cnibin\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.462929 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.460431 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b931b03f-d393-4723-b2fe-bc49c3475e67-cni-binary-copy\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.462929 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.460435 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/686793e7-d253-4516-9e2a-e53f8f528cf4-ovnkube-config\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.462929 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.460605 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b1e23db-0eff-418c-8f36-3e5dedc822ba-konnectivity-ca\") pod \"konnectivity-agent-9mt82\" (UID: \"4b1e23db-0eff-418c-8f36-3e5dedc822ba\") " pod="kube-system/konnectivity-agent-9mt82" Sep 15 07:18:41.462929 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.462415 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b1e23db-0eff-418c-8f36-3e5dedc822ba-agent-certs\") pod \"konnectivity-agent-9mt82\" (UID: \"4b1e23db-0eff-418c-8f36-3e5dedc822ba\") " pod="kube-system/konnectivity-agent-9mt82" Sep 15 07:18:41.463424 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.462940 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/686793e7-d253-4516-9e2a-e53f8f528cf4-ovn-node-metrics-cert\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.466419 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.466397 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrdnt\" (UniqueName: \"kubernetes.io/projected/686793e7-d253-4516-9e2a-e53f8f528cf4-kube-api-access-rrdnt\") pod \"ovnkube-node-xkh69\" (UID: \"686793e7-d253-4516-9e2a-e53f8f528cf4\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.469861 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.469815 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78x67\" (UniqueName: \"kubernetes.io/projected/b931b03f-d393-4723-b2fe-bc49c3475e67-kube-api-access-78x67\") pod \"multus-additional-cni-plugins-r2hmp\" (UID: \"b931b03f-d393-4723-b2fe-bc49c3475e67\") " pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.470191 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.470149 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb7l9\" (UniqueName: \"kubernetes.io/projected/b8eab75a-9838-4a5e-b7c9-b46b1d616807-kube-api-access-bb7l9\") pod \"iptables-alerter-r4mbg\" (UID: \"b8eab75a-9838-4a5e-b7c9-b46b1d616807\") " pod="openshift-network-operator/iptables-alerter-r4mbg" Sep 15 07:18:41.470377 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.470337 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6xpc\" (UniqueName: \"kubernetes.io/projected/b35488b0-f9fa-4e0b-bf35-87f7048b8a6a-kube-api-access-s6xpc\") pod \"aws-ebs-csi-driver-node-cjchv\" (UID: \"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.472609 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.472569 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" event={"ID":"ff4bf43fd141c579255e728a935942fa","Type":"ContainerStarted","Data":"0199abbeff2780baffc827cd1adae33cc281c32133937616d5ac04b4d6fe4660"} Sep 15 07:18:41.473755 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.473724 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" event={"ID":"438588d6d557fa6d076676dc45773c05","Type":"ContainerStarted","Data":"92fb934314879b68c2f9a34f643b1f29032f5c8673155cd268d449a5442856ea"} Sep 15 07:18:41.474132 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.474111 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htkll\" (UniqueName: \"kubernetes.io/projected/f13cc492-490d-478f-b799-6127572cc075-kube-api-access-htkll\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:41.561265 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561228 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-sys\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.561440 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561283 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/72e4df05-e7be-4167-b564-1fb5ccf015ce-multus-daemon-config\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561440 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561314 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-multus-cni-dir\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561440 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561340 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-var-lib-cni-bin\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561440 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561344 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-sys\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.561440 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561364 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxm2l\" (UniqueName: \"kubernetes.io/projected/72e4df05-e7be-4167-b564-1fb5ccf015ce-kube-api-access-jxm2l\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561440 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561403 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-kubernetes\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.561440 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561405 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-var-lib-cni-bin\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561440 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561435 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42649c54-356b-4265-9d26-2c42d9b0f265-tmp-dir\") pod \"node-resolver-xbtjp\" (UID: \"42649c54-356b-4265-9d26-2c42d9b0f265\") " pod="openshift-dns/node-resolver-xbtjp" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561452 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-multus-cni-dir\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561460 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72e4df05-e7be-4167-b564-1fb5ccf015ce-cni-binary-copy\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561503 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-run-netns\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561536 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-lib-modules\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561562 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-cnibin\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561571 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-kubernetes\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561587 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-run-multus-certs\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561612 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-systemd\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561616 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-run-netns\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561643 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-run\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561670 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-systemd\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561668 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-modprobe-d\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561707 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-var-lib-cni-multus\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561737 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-var-lib-kubelet\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561763 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-multus-conf-dir\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561785 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-lib-modules\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561791 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8428492-dcc5-4c33-bcd2-29002680f011-tmp\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.561927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561815 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfwb4\" (UniqueName: \"kubernetes.io/projected/7e2134f2-9ba8-432a-9ba9-aa8e313a86c6-kube-api-access-kfwb4\") pod \"node-ca-njnqn\" (UID: \"7e2134f2-9ba8-432a-9ba9-aa8e313a86c6\") " pod="openshift-image-registry/node-ca-njnqn" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561837 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/72e4df05-e7be-4167-b564-1fb5ccf015ce-multus-daemon-config\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561839 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-system-cni-dir\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561834 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-run-multus-certs\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561841 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-cnibin\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561890 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c8428492-dcc5-4c33-bcd2-29002680f011-etc-tuned\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561909 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-system-cni-dir\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561927 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtdg2\" (UniqueName: \"kubernetes.io/projected/42649c54-356b-4265-9d26-2c42d9b0f265-kube-api-access-mtdg2\") pod \"node-resolver-xbtjp\" (UID: \"42649c54-356b-4265-9d26-2c42d9b0f265\") " pod="openshift-dns/node-resolver-xbtjp" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561943 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e2134f2-9ba8-432a-9ba9-aa8e313a86c6-serviceca\") pod \"node-ca-njnqn\" (UID: \"7e2134f2-9ba8-432a-9ba9-aa8e313a86c6\") " pod="openshift-image-registry/node-ca-njnqn" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561958 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-run-k8s-cni-cncf-io\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561985 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f5nc\" (UniqueName: \"kubernetes.io/projected/c8428492-dcc5-4c33-bcd2-29002680f011-kube-api-access-6f5nc\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562005 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/42649c54-356b-4265-9d26-2c42d9b0f265-hosts-file\") pod \"node-resolver-xbtjp\" (UID: \"42649c54-356b-4265-9d26-2c42d9b0f265\") " pod="openshift-dns/node-resolver-xbtjp" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562031 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-var-lib-kubelet\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562038 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-multus-socket-dir-parent\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562072 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-sysconfig\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562082 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-multus-socket-dir-parent\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562099 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e2134f2-9ba8-432a-9ba9-aa8e313a86c6-host\") pod \"node-ca-njnqn\" (UID: \"7e2134f2-9ba8-432a-9ba9-aa8e313a86c6\") " pod="openshift-image-registry/node-ca-njnqn" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.561791 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42649c54-356b-4265-9d26-2c42d9b0f265-tmp-dir\") pod \"node-resolver-xbtjp\" (UID: \"42649c54-356b-4265-9d26-2c42d9b0f265\") " pod="openshift-dns/node-resolver-xbtjp" Sep 15 07:18:41.562759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562128 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlgn6\" (UniqueName: \"kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6\") pod \"network-check-target-vbjvp\" (UID: \"5ecac077-a8a5-4930-a69c-21402e834160\") " pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562154 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-etc-kubernetes\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562196 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-sysconfig\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562202 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-run\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562197 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72e4df05-e7be-4167-b564-1fb5ccf015ce-cni-binary-copy\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562234 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-sysctl-conf\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562265 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-run-k8s-cni-cncf-io\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562266 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-multus-conf-dir\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562276 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-var-lib-kubelet\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562308 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-host\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562335 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-sysctl-d\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562369 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-hostroot\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562431 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-hostroot\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562452 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e2134f2-9ba8-432a-9ba9-aa8e313a86c6-host\") pod \"node-ca-njnqn\" (UID: \"7e2134f2-9ba8-432a-9ba9-aa8e313a86c6\") " pod="openshift-image-registry/node-ca-njnqn" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562261 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-host-var-lib-cni-multus\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562573 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-sysctl-d\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562618 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-var-lib-kubelet\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562633 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-sysctl-conf\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.563298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562659 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e2134f2-9ba8-432a-9ba9-aa8e313a86c6-serviceca\") pod \"node-ca-njnqn\" (UID: \"7e2134f2-9ba8-432a-9ba9-aa8e313a86c6\") " pod="openshift-image-registry/node-ca-njnqn" Sep 15 07:18:41.563780 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562681 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/42649c54-356b-4265-9d26-2c42d9b0f265-hosts-file\") pod \"node-resolver-xbtjp\" (UID: \"42649c54-356b-4265-9d26-2c42d9b0f265\") " pod="openshift-dns/node-resolver-xbtjp" Sep 15 07:18:41.563780 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562691 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-host\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.563780 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562698 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72e4df05-e7be-4167-b564-1fb5ccf015ce-etc-kubernetes\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.563780 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.562002 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c8428492-dcc5-4c33-bcd2-29002680f011-etc-modprobe-d\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.564182 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.564163 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c8428492-dcc5-4c33-bcd2-29002680f011-etc-tuned\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.564270 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.564253 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8428492-dcc5-4c33-bcd2-29002680f011-tmp\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.567935 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.567915 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:18:41.567935 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.567933 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:18:41.568117 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.567943 2552 projected.go:196] Error preparing data for projected volume kube-api-access-qlgn6 for pod openshift-network-diagnostics/network-check-target-vbjvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:18:41.568117 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.568004 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6 podName:5ecac077-a8a5-4930-a69c-21402e834160 nodeName:}" failed. No retries permitted until 2026-09-15 07:18:42.067986904 +0000 UTC m=+3.049149069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qlgn6" (UniqueName: "kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6") pod "network-check-target-vbjvp" (UID: "5ecac077-a8a5-4930-a69c-21402e834160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:18:41.570131 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.570106 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxm2l\" (UniqueName: \"kubernetes.io/projected/72e4df05-e7be-4167-b564-1fb5ccf015ce-kube-api-access-jxm2l\") pod \"multus-djmmr\" (UID: \"72e4df05-e7be-4167-b564-1fb5ccf015ce\") " pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.570240 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.570185 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f5nc\" (UniqueName: \"kubernetes.io/projected/c8428492-dcc5-4c33-bcd2-29002680f011-kube-api-access-6f5nc\") pod \"tuned-dfrxr\" (UID: \"c8428492-dcc5-4c33-bcd2-29002680f011\") " pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.570823 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.570802 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtdg2\" (UniqueName: \"kubernetes.io/projected/42649c54-356b-4265-9d26-2c42d9b0f265-kube-api-access-mtdg2\") pod \"node-resolver-xbtjp\" (UID: \"42649c54-356b-4265-9d26-2c42d9b0f265\") " pod="openshift-dns/node-resolver-xbtjp" Sep 15 07:18:41.571266 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.571237 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfwb4\" (UniqueName: \"kubernetes.io/projected/7e2134f2-9ba8-432a-9ba9-aa8e313a86c6-kube-api-access-kfwb4\") pod \"node-ca-njnqn\" (UID: \"7e2134f2-9ba8-432a-9ba9-aa8e313a86c6\") " pod="openshift-image-registry/node-ca-njnqn" Sep 15 07:18:41.651334 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.651237 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" Sep 15 07:18:41.653711 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.653681 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 07:18:41.660998 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.660969 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r4mbg" Sep 15 07:18:41.669729 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.669710 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:18:41.675742 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.675716 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9mt82" Sep 15 07:18:41.683247 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.683224 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-r2hmp" Sep 15 07:18:41.689954 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.689935 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" Sep 15 07:18:41.697561 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.697543 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xbtjp" Sep 15 07:18:41.705048 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.705025 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-djmmr" Sep 15 07:18:41.711478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.711462 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-njnqn" Sep 15 07:18:41.965729 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.965653 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:41.965729 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:41.965703 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:41.965925 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.965840 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:18:41.965925 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.965910 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs podName:f13cc492-490d-478f-b799-6127572cc075 nodeName:}" failed. No retries permitted until 2026-09-15 07:18:42.965893546 +0000 UTC m=+3.947055691 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs") pod "network-metrics-daemon-n9rxl" (UID: "f13cc492-490d-478f-b799-6127572cc075") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:18:41.965925 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.965840 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:18:41.966059 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:41.965979 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret podName:c6f6820e-f789-494a-9a66-fe16e0c2f21a nodeName:}" failed. No retries permitted until 2026-09-15 07:18:42.965965613 +0000 UTC m=+3.947127775 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret") pod "global-pull-secret-syncer-h8b5m" (UID: "c6f6820e-f789-494a-9a66-fe16e0c2f21a") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:18:42.166930 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.166897 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlgn6\" (UniqueName: \"kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6\") pod \"network-check-target-vbjvp\" (UID: \"5ecac077-a8a5-4930-a69c-21402e834160\") " pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:42.167095 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:42.167051 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:18:42.167095 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:42.167071 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:18:42.167095 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:42.167082 2552 projected.go:196] Error preparing data for projected volume kube-api-access-qlgn6 for pod openshift-network-diagnostics/network-check-target-vbjvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:18:42.167274 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:42.167133 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6 podName:5ecac077-a8a5-4930-a69c-21402e834160 nodeName:}" failed. No retries permitted until 2026-09-15 07:18:43.167119202 +0000 UTC m=+4.148281363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlgn6" (UniqueName: "kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6") pod "network-check-target-vbjvp" (UID: "5ecac077-a8a5-4930-a69c-21402e834160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:18:42.195029 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:42.194862 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e4df05_e7be_4167_b564_1fb5ccf015ce.slice/crio-00c1d90572369515ce0f3dcb1d9d5bd14009ae314a44f6d3695fbee428553a27 WatchSource:0}: Error finding container 00c1d90572369515ce0f3dcb1d9d5bd14009ae314a44f6d3695fbee428553a27: Status 404 returned error can't find the container with id 00c1d90572369515ce0f3dcb1d9d5bd14009ae314a44f6d3695fbee428553a27 Sep 15 07:18:42.197124 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:42.197091 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42649c54_356b_4265_9d26_2c42d9b0f265.slice/crio-76efd0f6bee0ef036cbd0ec340a7e1769a2a6962b91535e0486e9bddc369bd7f WatchSource:0}: Error finding container 76efd0f6bee0ef036cbd0ec340a7e1769a2a6962b91535e0486e9bddc369bd7f: Status 404 returned error can't find the container with id 76efd0f6bee0ef036cbd0ec340a7e1769a2a6962b91535e0486e9bddc369bd7f Sep 15 07:18:42.198236 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:42.198167 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8eab75a_9838_4a5e_b7c9_b46b1d616807.slice/crio-a22d862c85859c8fb9508e8a3b04d86c7763edd44ee0c76d770a562f75a969d9 WatchSource:0}: Error finding container a22d862c85859c8fb9508e8a3b04d86c7763edd44ee0c76d770a562f75a969d9: Status 404 returned error can't find the container with id a22d862c85859c8fb9508e8a3b04d86c7763edd44ee0c76d770a562f75a969d9 Sep 15 07:18:42.199573 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:42.199384 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb35488b0_f9fa_4e0b_bf35_87f7048b8a6a.slice/crio-cf6a8280ce4dd2ac811d5e419f9954a7953e9279af3cf799126ebb0bc1bf1d0a WatchSource:0}: Error finding container cf6a8280ce4dd2ac811d5e419f9954a7953e9279af3cf799126ebb0bc1bf1d0a: Status 404 returned error can't find the container with id cf6a8280ce4dd2ac811d5e419f9954a7953e9279af3cf799126ebb0bc1bf1d0a Sep 15 07:18:42.201808 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:42.201760 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod686793e7_d253_4516_9e2a_e53f8f528cf4.slice/crio-866ff34ff77cd40b91d9d925e60d2beec4a8ccbbe6043fd90f505ebb3c66dd9e WatchSource:0}: Error finding container 866ff34ff77cd40b91d9d925e60d2beec4a8ccbbe6043fd90f505ebb3c66dd9e: Status 404 returned error can't find the container with id 866ff34ff77cd40b91d9d925e60d2beec4a8ccbbe6043fd90f505ebb3c66dd9e Sep 15 07:18:42.202500 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:42.202481 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8428492_dcc5_4c33_bcd2_29002680f011.slice/crio-d69e794555ace5d8290e41645083d50299a963cb6f8e0d8575f09a5660be47e5 WatchSource:0}: Error finding container d69e794555ace5d8290e41645083d50299a963cb6f8e0d8575f09a5660be47e5: Status 404 returned error can't find the container with id d69e794555ace5d8290e41645083d50299a963cb6f8e0d8575f09a5660be47e5 Sep 15 07:18:42.204238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:42.204160 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e2134f2_9ba8_432a_9ba9_aa8e313a86c6.slice/crio-3e5cf6d2fc6faae3d6e827da7298fc1eeee28f698afb129a27166e293f753b97 WatchSource:0}: Error finding container 3e5cf6d2fc6faae3d6e827da7298fc1eeee28f698afb129a27166e293f753b97: Status 404 returned error can't find the container with id 3e5cf6d2fc6faae3d6e827da7298fc1eeee28f698afb129a27166e293f753b97 Sep 15 07:18:42.205038 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:42.204996 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb931b03f_d393_4723_b2fe_bc49c3475e67.slice/crio-75faff1657b1d252c42080f678f69efdd0ea80dd9129b906be38157c8065ea9b WatchSource:0}: Error finding container 75faff1657b1d252c42080f678f69efdd0ea80dd9129b906be38157c8065ea9b: Status 404 returned error can't find the container with id 75faff1657b1d252c42080f678f69efdd0ea80dd9129b906be38157c8065ea9b Sep 15 07:18:42.206736 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:18:42.206713 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b1e23db_0eff_418c_8f36_3e5dedc822ba.slice/crio-c23b6c51036340b292ed4b5cae758e93caa63447308bd10e4ce6290f812516ce WatchSource:0}: Error finding container c23b6c51036340b292ed4b5cae758e93caa63447308bd10e4ce6290f812516ce: Status 404 returned error can't find the container with id c23b6c51036340b292ed4b5cae758e93caa63447308bd10e4ce6290f812516ce Sep 15 07:18:42.387165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.387100 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 07:13:41 +0000 UTC" deadline="2028-03-14 07:37:20.108703939 +0000 UTC" Sep 15 07:18:42.387673 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.387171 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13104h18m37.721536948s" Sep 15 07:18:42.468131 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.468102 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:42.468299 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:42.468238 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:18:42.477476 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.477409 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9mt82" event={"ID":"4b1e23db-0eff-418c-8f36-3e5dedc822ba","Type":"ContainerStarted","Data":"c23b6c51036340b292ed4b5cae758e93caa63447308bd10e4ce6290f812516ce"} Sep 15 07:18:42.478743 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.478716 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" event={"ID":"686793e7-d253-4516-9e2a-e53f8f528cf4","Type":"ContainerStarted","Data":"866ff34ff77cd40b91d9d925e60d2beec4a8ccbbe6043fd90f505ebb3c66dd9e"} Sep 15 07:18:42.479843 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.479810 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" event={"ID":"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a","Type":"ContainerStarted","Data":"cf6a8280ce4dd2ac811d5e419f9954a7953e9279af3cf799126ebb0bc1bf1d0a"} Sep 15 07:18:42.481069 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.481043 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xbtjp" event={"ID":"42649c54-356b-4265-9d26-2c42d9b0f265","Type":"ContainerStarted","Data":"76efd0f6bee0ef036cbd0ec340a7e1769a2a6962b91535e0486e9bddc369bd7f"} Sep 15 07:18:42.482320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.482295 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-djmmr" event={"ID":"72e4df05-e7be-4167-b564-1fb5ccf015ce","Type":"ContainerStarted","Data":"00c1d90572369515ce0f3dcb1d9d5bd14009ae314a44f6d3695fbee428553a27"} Sep 15 07:18:42.484594 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.484562 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" event={"ID":"ff4bf43fd141c579255e728a935942fa","Type":"ContainerStarted","Data":"34bec6cfe251f5efd7f5ebde2c3e4e3b3eb5b7479fbaf125f4198d326e91f9ac"} Sep 15 07:18:42.485920 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.485897 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r2hmp" event={"ID":"b931b03f-d393-4723-b2fe-bc49c3475e67","Type":"ContainerStarted","Data":"75faff1657b1d252c42080f678f69efdd0ea80dd9129b906be38157c8065ea9b"} Sep 15 07:18:42.487230 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.487132 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-njnqn" event={"ID":"7e2134f2-9ba8-432a-9ba9-aa8e313a86c6","Type":"ContainerStarted","Data":"3e5cf6d2fc6faae3d6e827da7298fc1eeee28f698afb129a27166e293f753b97"} Sep 15 07:18:42.488348 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.488325 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" event={"ID":"c8428492-dcc5-4c33-bcd2-29002680f011","Type":"ContainerStarted","Data":"d69e794555ace5d8290e41645083d50299a963cb6f8e0d8575f09a5660be47e5"} Sep 15 07:18:42.489499 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.489480 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r4mbg" event={"ID":"b8eab75a-9838-4a5e-b7c9-b46b1d616807","Type":"ContainerStarted","Data":"a22d862c85859c8fb9508e8a3b04d86c7763edd44ee0c76d770a562f75a969d9"} Sep 15 07:18:42.497467 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.497416 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-175.ec2.internal" podStartSLOduration=2.49739953 podStartE2EDuration="2.49739953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:18:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:18:42.49666319 +0000 UTC m=+3.477825357" watchObservedRunningTime="2026-09-15 07:18:42.49739953 +0000 UTC m=+3.478561700" Sep 15 07:18:42.974845 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.973951 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:42.974845 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:42.974044 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:42.974845 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:42.974185 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:18:42.974845 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:42.974256 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret podName:c6f6820e-f789-494a-9a66-fe16e0c2f21a nodeName:}" failed. No retries permitted until 2026-09-15 07:18:44.974237822 +0000 UTC m=+5.955399970 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret") pod "global-pull-secret-syncer-h8b5m" (UID: "c6f6820e-f789-494a-9a66-fe16e0c2f21a") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:18:42.974845 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:42.974688 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:18:42.974845 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:42.974740 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs podName:f13cc492-490d-478f-b799-6127572cc075 nodeName:}" failed. No retries permitted until 2026-09-15 07:18:44.974724687 +0000 UTC m=+5.955886836 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs") pod "network-metrics-daemon-n9rxl" (UID: "f13cc492-490d-478f-b799-6127572cc075") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:18:43.176532 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:43.176421 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlgn6\" (UniqueName: \"kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6\") pod \"network-check-target-vbjvp\" (UID: \"5ecac077-a8a5-4930-a69c-21402e834160\") " pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:43.176699 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:43.176630 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:18:43.176699 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:43.176657 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:18:43.176699 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:43.176671 2552 projected.go:196] Error preparing data for projected volume kube-api-access-qlgn6 for pod openshift-network-diagnostics/network-check-target-vbjvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:18:43.176861 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:43.176754 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6 podName:5ecac077-a8a5-4930-a69c-21402e834160 nodeName:}" failed. No retries permitted until 2026-09-15 07:18:45.176735379 +0000 UTC m=+6.157897544 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlgn6" (UniqueName: "kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6") pod "network-check-target-vbjvp" (UID: "5ecac077-a8a5-4930-a69c-21402e834160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:18:43.387851 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:43.387751 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 07:13:41 +0000 UTC" deadline="2028-03-15 10:15:11.378314316 +0000 UTC" Sep 15 07:18:43.387851 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:43.387794 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13130h56m27.990524922s" Sep 15 07:18:43.469490 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:43.468988 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:43.469490 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:43.468998 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:43.469490 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:43.469131 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:18:43.469490 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:43.469188 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:18:44.468140 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:44.468086 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:44.468524 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:44.468306 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:18:44.998118 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:44.997244 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:44.998118 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:44.997378 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:44.998118 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:44.997520 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:18:44.998118 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:44.997584 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret podName:c6f6820e-f789-494a-9a66-fe16e0c2f21a nodeName:}" failed. No retries permitted until 2026-09-15 07:18:48.997566167 +0000 UTC m=+9.978728315 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret") pod "global-pull-secret-syncer-h8b5m" (UID: "c6f6820e-f789-494a-9a66-fe16e0c2f21a") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:18:44.998118 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:44.998000 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:18:44.998118 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:44.998048 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs podName:f13cc492-490d-478f-b799-6127572cc075 nodeName:}" failed. No retries permitted until 2026-09-15 07:18:48.998033074 +0000 UTC m=+9.979195224 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs") pod "network-metrics-daemon-n9rxl" (UID: "f13cc492-490d-478f-b799-6127572cc075") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:18:45.199567 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:45.199496 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlgn6\" (UniqueName: \"kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6\") pod \"network-check-target-vbjvp\" (UID: \"5ecac077-a8a5-4930-a69c-21402e834160\") " pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:45.199760 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:45.199714 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:18:45.199760 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:45.199736 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:18:45.199760 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:45.199749 2552 projected.go:196] Error preparing data for projected volume kube-api-access-qlgn6 for pod openshift-network-diagnostics/network-check-target-vbjvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:18:45.199946 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:45.199816 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6 podName:5ecac077-a8a5-4930-a69c-21402e834160 nodeName:}" failed. No retries permitted until 2026-09-15 07:18:49.19979699 +0000 UTC m=+10.180959139 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlgn6" (UniqueName: "kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6") pod "network-check-target-vbjvp" (UID: "5ecac077-a8a5-4930-a69c-21402e834160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:18:45.468807 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:45.468451 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:45.468807 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:45.468578 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:45.468807 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:45.468596 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:18:45.468807 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:45.468696 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:18:46.468233 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:46.468197 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:46.468427 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:46.468352 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:18:47.468505 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:47.468344 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:47.468505 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:47.468397 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:47.468505 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:47.468480 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:18:47.469132 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:47.468599 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:18:48.468938 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:48.468848 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:48.469489 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:48.468983 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:18:49.035119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:49.035079 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:49.035317 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:49.035173 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:49.035317 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:49.035284 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:18:49.035317 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:49.035311 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:18:49.035485 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:49.035380 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret podName:c6f6820e-f789-494a-9a66-fe16e0c2f21a nodeName:}" failed. No retries permitted until 2026-09-15 07:18:57.035361343 +0000 UTC m=+18.016523500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret") pod "global-pull-secret-syncer-h8b5m" (UID: "c6f6820e-f789-494a-9a66-fe16e0c2f21a") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:18:49.035485 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:49.035421 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs podName:f13cc492-490d-478f-b799-6127572cc075 nodeName:}" failed. No retries permitted until 2026-09-15 07:18:57.035403125 +0000 UTC m=+18.016565289 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs") pod "network-metrics-daemon-n9rxl" (UID: "f13cc492-490d-478f-b799-6127572cc075") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:18:49.236017 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:49.235987 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlgn6\" (UniqueName: \"kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6\") pod \"network-check-target-vbjvp\" (UID: \"5ecac077-a8a5-4930-a69c-21402e834160\") " pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:49.236192 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:49.236174 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:18:49.236242 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:49.236199 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:18:49.236242 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:49.236214 2552 projected.go:196] Error preparing data for projected volume kube-api-access-qlgn6 for pod openshift-network-diagnostics/network-check-target-vbjvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:18:49.236341 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:49.236280 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6 podName:5ecac077-a8a5-4930-a69c-21402e834160 nodeName:}" failed. No retries permitted until 2026-09-15 07:18:57.23626515 +0000 UTC m=+18.217427300 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlgn6" (UniqueName: "kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6") pod "network-check-target-vbjvp" (UID: "5ecac077-a8a5-4930-a69c-21402e834160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:18:49.469676 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:49.469595 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:49.470152 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:49.469700 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:18:49.470345 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:49.470326 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:49.470458 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:49.470431 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:18:50.468401 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:50.468368 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:50.468577 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:50.468496 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:18:51.468520 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:51.468489 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:51.468856 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:51.468621 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:18:51.468856 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:51.468693 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:51.468856 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:51.468782 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:18:52.468811 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:52.468780 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:52.469211 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:52.468918 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:18:53.468496 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:53.468464 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:53.468496 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:53.468500 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:53.468700 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:53.468583 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:18:53.468759 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:53.468717 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:18:54.468638 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:54.468609 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:54.469109 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:54.468715 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:18:55.468538 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:55.468504 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:55.468711 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:55.468550 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:55.468711 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:55.468629 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:18:55.469192 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:55.468751 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:18:56.468674 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:56.468593 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:56.468852 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:56.468757 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:18:57.093802 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:57.093770 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:57.094003 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:57.093849 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:57.094003 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:57.093956 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:18:57.094121 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:57.094008 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:18:57.094121 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:57.094029 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs podName:f13cc492-490d-478f-b799-6127572cc075 nodeName:}" failed. No retries permitted until 2026-09-15 07:19:13.094008101 +0000 UTC m=+34.075170250 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs") pod "network-metrics-daemon-n9rxl" (UID: "f13cc492-490d-478f-b799-6127572cc075") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:18:57.094121 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:57.094060 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret podName:c6f6820e-f789-494a-9a66-fe16e0c2f21a nodeName:}" failed. No retries permitted until 2026-09-15 07:19:13.094049505 +0000 UTC m=+34.075211667 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret") pod "global-pull-secret-syncer-h8b5m" (UID: "c6f6820e-f789-494a-9a66-fe16e0c2f21a") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:18:57.294851 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:57.294819 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlgn6\" (UniqueName: \"kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6\") pod \"network-check-target-vbjvp\" (UID: \"5ecac077-a8a5-4930-a69c-21402e834160\") " pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:57.295054 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:57.295033 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:18:57.295109 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:57.295066 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:18:57.295109 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:57.295079 2552 projected.go:196] Error preparing data for projected volume kube-api-access-qlgn6 for pod openshift-network-diagnostics/network-check-target-vbjvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:18:57.295194 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:57.295139 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6 podName:5ecac077-a8a5-4930-a69c-21402e834160 nodeName:}" failed. No retries permitted until 2026-09-15 07:19:13.29512082 +0000 UTC m=+34.276282980 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlgn6" (UniqueName: "kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6") pod "network-check-target-vbjvp" (UID: "5ecac077-a8a5-4930-a69c-21402e834160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:18:57.468389 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:57.468303 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:57.468389 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:57.468328 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:57.468572 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:57.468447 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:18:57.468572 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:57.468555 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:18:58.468886 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:58.468848 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:18:58.469344 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:58.469013 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:18:59.469580 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:59.469550 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:18:59.469945 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:59.469630 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:18:59.469945 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:18:59.469703 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:18:59.469945 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:18:59.469796 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:19:00.468351 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.468053 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:19:00.468505 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:00.468428 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:19:00.535646 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.535561 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" event={"ID":"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a","Type":"ContainerStarted","Data":"26555f9244d3cb773822f5b3086349a9fb266e744e0f980bd0f103cec2242659"} Sep 15 07:19:00.536991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.536965 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xbtjp" event={"ID":"42649c54-356b-4265-9d26-2c42d9b0f265","Type":"ContainerStarted","Data":"9858ea93754dc3192561ad10e0788634571c7f6d2fa620e76a760a06050c3f69"} Sep 15 07:19:00.538438 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.538419 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-djmmr" event={"ID":"72e4df05-e7be-4167-b564-1fb5ccf015ce","Type":"ContainerStarted","Data":"594ea1d5a5d6a35d88c02c1a527793586aa96154b6992b47ca465298d8d9bed7"} Sep 15 07:19:00.539959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.539937 2552 generic.go:356] "Generic (PLEG): container finished" podID="438588d6d557fa6d076676dc45773c05" containerID="76bd964465c5c3d7302605c4c5caba014d34a4fd65247a4ce97e58c23489eb21" exitCode=0 Sep 15 07:19:00.540049 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.539964 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" event={"ID":"438588d6d557fa6d076676dc45773c05","Type":"ContainerDied","Data":"76bd964465c5c3d7302605c4c5caba014d34a4fd65247a4ce97e58c23489eb21"} Sep 15 07:19:00.541600 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.541582 2552 generic.go:356] "Generic (PLEG): container finished" podID="b931b03f-d393-4723-b2fe-bc49c3475e67" containerID="a387f4ea511376403747e4dbd370f4a5b92ec4ccd3cf2bebb59c87c70763b0f3" exitCode=0 Sep 15 07:19:00.541697 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.541623 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r2hmp" event={"ID":"b931b03f-d393-4723-b2fe-bc49c3475e67","Type":"ContainerDied","Data":"a387f4ea511376403747e4dbd370f4a5b92ec4ccd3cf2bebb59c87c70763b0f3"} Sep 15 07:19:00.543289 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.543227 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-njnqn" event={"ID":"7e2134f2-9ba8-432a-9ba9-aa8e313a86c6","Type":"ContainerStarted","Data":"80b38721b83dd063ca69a7874fdbb8f7de19501058482880c7e487c2ac6417bf"} Sep 15 07:19:00.544765 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.544746 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" event={"ID":"c8428492-dcc5-4c33-bcd2-29002680f011","Type":"ContainerStarted","Data":"dfc7d1af9c3a6b8e25cf06329a140a9f7016945346ebdcf631740334291a96a9"} Sep 15 07:19:00.546118 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.546089 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9mt82" event={"ID":"4b1e23db-0eff-418c-8f36-3e5dedc822ba","Type":"ContainerStarted","Data":"ad49a574d6c4a339596bfa7507864feafba5b06bad21a750a45bace198ec31dd"} Sep 15 07:19:00.549258 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.549233 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:19:00.549617 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.549598 2552 generic.go:356] "Generic (PLEG): container finished" podID="686793e7-d253-4516-9e2a-e53f8f528cf4" containerID="8c42dada54b1b48dc872664d8b70ff6bf5558ad5f85749b899376c07ccb38ea8" exitCode=1 Sep 15 07:19:00.549690 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.549629 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" event={"ID":"686793e7-d253-4516-9e2a-e53f8f528cf4","Type":"ContainerStarted","Data":"996c0dcaea729765338357c2cc6e635b42808926de5125f50d32c21499fcff41"} Sep 15 07:19:00.549726 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.549690 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" event={"ID":"686793e7-d253-4516-9e2a-e53f8f528cf4","Type":"ContainerStarted","Data":"5b897f95a3edcbe17977fc1636e3af99d59e0fb5c64116d7af5e9059d9bfc2ba"} Sep 15 07:19:00.549770 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.549740 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" event={"ID":"686793e7-d253-4516-9e2a-e53f8f528cf4","Type":"ContainerStarted","Data":"1951a034893f04ccbe9f3d86678edea2feb3b70ed943486744c8ce316b534197"} Sep 15 07:19:00.549807 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.549787 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" event={"ID":"686793e7-d253-4516-9e2a-e53f8f528cf4","Type":"ContainerStarted","Data":"d58a945b297367b8604bbdf4a25df54e286e6e7a8c4089bd51d6a1c43fd7e04d"} Sep 15 07:19:00.549844 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.549833 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" event={"ID":"686793e7-d253-4516-9e2a-e53f8f528cf4","Type":"ContainerDied","Data":"8c42dada54b1b48dc872664d8b70ff6bf5558ad5f85749b899376c07ccb38ea8"} Sep 15 07:19:00.549910 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.549898 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" event={"ID":"686793e7-d253-4516-9e2a-e53f8f528cf4","Type":"ContainerStarted","Data":"8f7cacf67e263a1c8ba1cb937048336c4b7968d364f59b6727a478e00a17278e"} Sep 15 07:19:00.551635 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.551604 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xbtjp" podStartSLOduration=4.155579507 podStartE2EDuration="21.55157961s" totalImagesPullingTime="17.396000103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:00.551481894 +0000 UTC m=+21.532644062" watchObservedRunningTime="2026-09-15 07:19:00.55157961 +0000 UTC m=+21.532741771" Sep 15 07:19:00.567191 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.567153 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-djmmr" podStartSLOduration=3.781883799 podStartE2EDuration="21.567140727s" totalImagesPullingTime="17.785256928s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:00.566697961 +0000 UTC m=+21.547860151" watchObservedRunningTime="2026-09-15 07:19:00.567140727 +0000 UTC m=+21.548302893" Sep 15 07:19:00.579811 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.579771 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9mt82" podStartSLOduration=7.742217843 podStartE2EDuration="21.579758137s" totalImagesPullingTime="13.837540294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:00.579457289 +0000 UTC m=+21.560619457" watchObservedRunningTime="2026-09-15 07:19:00.579758137 +0000 UTC m=+21.560920301" Sep 15 07:19:00.617476 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.617440 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dfrxr" podStartSLOduration=4.226229984 podStartE2EDuration="21.617429146s" totalImagesPullingTime="17.391199162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:00.616983222 +0000 UTC m=+21.598145390" watchObservedRunningTime="2026-09-15 07:19:00.617429146 +0000 UTC m=+21.598591312" Sep 15 07:19:00.633393 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.633331 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-njnqn" podStartSLOduration=4.261373019 podStartE2EDuration="21.633311426s" totalImagesPullingTime="17.371938407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:00.632826719 +0000 UTC m=+21.613988944" watchObservedRunningTime="2026-09-15 07:19:00.633311426 +0000 UTC m=+21.614473610" Sep 15 07:19:00.671301 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.671230 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:00.753694 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:00.753662 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:01.211626 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.211485 2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 15 07:19:01.410283 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.410098 2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T07:19:01.211618713Z","UUID":"fb860667-7e06-44a4-9ab1-4437e856994d","Handler":null,"Name":"","Endpoint":""} Sep 15 07:19:01.411766 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.411746 2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 15 07:19:01.411904 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.411778 2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 15 07:19:01.472481 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.468972 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:19:01.472481 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:01.469147 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:19:01.472481 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.469192 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:01.472481 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:01.469335 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:19:01.554534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.554501 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" event={"ID":"438588d6d557fa6d076676dc45773c05","Type":"ContainerStarted","Data":"44b3eff5fd34b26788af4da43819e3b73299c8bd0b35a151a43f8a2c3d217b5e"} Sep 15 07:19:01.556172 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.556142 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r4mbg" event={"ID":"b8eab75a-9838-4a5e-b7c9-b46b1d616807","Type":"ContainerStarted","Data":"b69ca5b0c16bf996c3eda482e713955811073486034e73a637de5bc299564c78"} Sep 15 07:19:01.558173 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.558144 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" event={"ID":"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a","Type":"ContainerStarted","Data":"13dd26aee36bbba7d271e1512d2e07a89d2ed1ac08b90b6c0d2d3fa0f77d10e7"} Sep 15 07:19:01.569695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.569655 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-175.ec2.internal" podStartSLOduration=21.569642529 podStartE2EDuration="21.569642529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:18:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:01.569142364 +0000 UTC m=+22.550304532" watchObservedRunningTime="2026-09-15 07:19:01.569642529 +0000 UTC m=+22.550804695" Sep 15 07:19:01.582117 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.582064 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r4mbg" podStartSLOduration=5.184700679 podStartE2EDuration="22.582047758s" totalImagesPullingTime="17.397347079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:01.58176508 +0000 UTC m=+22.562927260" watchObservedRunningTime="2026-09-15 07:19:01.582047758 +0000 UTC m=+22.563209925" Sep 15 07:19:01.697695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.697627 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:01.783891 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:01.783843 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:02.453840 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:02.453767 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9mt82" Sep 15 07:19:02.454486 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:02.454445 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9mt82" Sep 15 07:19:02.468093 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:02.468066 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:19:02.468273 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:02.468183 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:19:02.564923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:02.564899 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:19:02.565509 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:02.565305 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" event={"ID":"686793e7-d253-4516-9e2a-e53f8f528cf4","Type":"ContainerStarted","Data":"7001efe7346472c1d067fc539bed723f220ca475dad44977c956e9fcb1ff48d6"} Sep 15 07:19:02.567671 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:02.567639 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" event={"ID":"b35488b0-f9fa-4e0b-bf35-87f7048b8a6a","Type":"ContainerStarted","Data":"61afba45be99df430d54321095197255c854c23cba28f09c115c5c509926eacf"} Sep 15 07:19:02.568461 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:02.568197 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9mt82" Sep 15 07:19:02.568461 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:02.568457 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9mt82" Sep 15 07:19:02.586973 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:02.586931 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cjchv" podStartSLOduration=4.391220345 podStartE2EDuration="23.586913731s" totalImagesPullingTime="19.195693386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:18:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:02.586244377 +0000 UTC m=+23.567406545" watchObservedRunningTime="2026-09-15 07:19:02.586913731 +0000 UTC m=+23.568075899" Sep 15 07:19:02.642532 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:02.642499 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:03.469028 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:03.468742 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:03.469028 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:03.468901 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:19:03.469028 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:03.468941 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:19:03.469403 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:03.469066 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:19:04.407988 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:04.407953 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:04.468509 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:04.468475 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:19:04.468669 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:04.468647 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:19:04.470788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:04.470759 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:05.469037 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:05.468838 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:05.469379 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:05.468916 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:19:05.469379 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:05.469073 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:19:05.469379 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:05.469189 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:19:05.578529 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:05.578503 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:19:05.579138 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:05.579112 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" event={"ID":"686793e7-d253-4516-9e2a-e53f8f528cf4","Type":"ContainerStarted","Data":"37472fb26e72dc6ed74fb0ec2c3107eb3f2168a14e2721a0ccca89cc0cd9dbf5"} Sep 15 07:19:05.579379 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:05.579352 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:19:05.579550 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:05.579536 2552 scope.go:117] "RemoveContainer" containerID="8c42dada54b1b48dc872664d8b70ff6bf5558ad5f85749b899376c07ccb38ea8" Sep 15 07:19:05.581126 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:05.581109 2552 generic.go:356] "Generic (PLEG): container finished" podID="b931b03f-d393-4723-b2fe-bc49c3475e67" containerID="83b10597d50a54650bcaddbbbade13a7bccc7334eb6edb4cb892cdcc819963fd" exitCode=0 Sep 15 07:19:05.581215 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:05.581139 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r2hmp" event={"ID":"b931b03f-d393-4723-b2fe-bc49c3475e67","Type":"ContainerDied","Data":"83b10597d50a54650bcaddbbbade13a7bccc7334eb6edb4cb892cdcc819963fd"} Sep 15 07:19:05.595831 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:05.595811 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:19:05.848353 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:05.848319 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:06.035228 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:06.035190 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:06.468694 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:06.468667 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:19:06.468807 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:06.468784 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:19:06.585778 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:06.585743 2552 generic.go:356] "Generic (PLEG): container finished" podID="b931b03f-d393-4723-b2fe-bc49c3475e67" containerID="b79cf23da3894a5752f63b9d1a93916b79a1671be1413f6e1dcb30e69973c717" exitCode=0 Sep 15 07:19:06.586333 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:06.585816 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r2hmp" event={"ID":"b931b03f-d393-4723-b2fe-bc49c3475e67","Type":"ContainerDied","Data":"b79cf23da3894a5752f63b9d1a93916b79a1671be1413f6e1dcb30e69973c717"} Sep 15 07:19:06.590143 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:06.590121 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:19:06.590520 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:06.590498 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" event={"ID":"686793e7-d253-4516-9e2a-e53f8f528cf4","Type":"ContainerStarted","Data":"adb7cfe59c07bb8ead067610ee4c47f3619ac7811ef3e5decb2732c6c211d20d"} Sep 15 07:19:06.590828 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:06.590801 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:19:06.590925 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:06.590908 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:19:06.604525 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:06.604507 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:19:06.658962 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:06.658856 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" podStartSLOduration=10.226701861 podStartE2EDuration="27.658840408s" totalImagesPullingTime="17.432138547s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:06.657258573 +0000 UTC m=+27.638420779" watchObservedRunningTime="2026-09-15 07:19:06.658840408 +0000 UTC m=+27.640002572" Sep 15 07:19:07.110517 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:07.110484 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h8b5m"] Sep 15 07:19:07.110681 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:07.110609 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:19:07.110747 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:07.110710 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:19:07.113559 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:07.113532 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vbjvp"] Sep 15 07:19:07.113678 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:07.113656 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:07.113781 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:07.113760 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:19:07.114322 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:07.114285 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n9rxl"] Sep 15 07:19:07.114424 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:07.114384 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:19:07.114516 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:07.114490 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:19:07.594909 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:07.594861 2552 generic.go:356] "Generic (PLEG): container finished" podID="b931b03f-d393-4723-b2fe-bc49c3475e67" containerID="2a192a092b80857b210107645e513366408623814ee6034cb61ffe33437df92e" exitCode=0 Sep 15 07:19:07.595461 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:07.594963 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r2hmp" event={"ID":"b931b03f-d393-4723-b2fe-bc49c3475e67","Type":"ContainerDied","Data":"2a192a092b80857b210107645e513366408623814ee6034cb61ffe33437df92e"} Sep 15 07:19:08.468912 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:08.468601 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:19:08.469056 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:08.468627 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:08.469056 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:08.468628 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:19:08.469056 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:08.468975 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:19:08.469214 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:08.469074 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:19:08.469214 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:08.469147 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:19:10.468439 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:10.468407 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:19:10.468800 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:10.468456 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:10.468800 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:10.468531 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:19:10.468800 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:10.468545 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:19:10.468800 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:10.468599 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:19:10.468800 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:10.468650 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:19:10.798106 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:10.798069 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:12.468577 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.468551 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:19:12.468852 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.468592 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:12.468852 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.468639 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:19:12.468852 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:12.468656 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h8b5m" podUID="c6f6820e-f789-494a-9a66-fe16e0c2f21a" Sep 15 07:19:12.468852 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:12.468714 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:19:12.468852 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:12.468772 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbjvp" podUID="5ecac077-a8a5-4930-a69c-21402e834160" Sep 15 07:19:12.609386 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.609313 2552 generic.go:356] "Generic (PLEG): container finished" podID="b931b03f-d393-4723-b2fe-bc49c3475e67" containerID="c594b8ad25903158de4bf105e19bd274f59723c001a71a971360f710a17ff86d" exitCode=0 Sep 15 07:19:12.609386 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.609376 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r2hmp" event={"ID":"b931b03f-d393-4723-b2fe-bc49c3475e67","Type":"ContainerDied","Data":"c594b8ad25903158de4bf105e19bd274f59723c001a71a971360f710a17ff86d"} Sep 15 07:19:12.730465 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.730432 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:12.755445 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.755427 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-175.ec2.internal" event="NodeReady" Sep 15 07:19:12.755587 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.755576 2552 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 15 07:19:12.798910 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.798886 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bsrr6"] Sep 15 07:19:12.807802 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.807779 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:12.808713 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.808696 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bdtkq"] Sep 15 07:19:12.808826 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.808811 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:12.811674 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.811634 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tfj9f\"" Sep 15 07:19:12.811809 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.811689 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 15 07:19:12.811932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.811916 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 15 07:19:12.820692 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.820669 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bdtkq" Sep 15 07:19:12.820804 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.820765 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bsrr6"] Sep 15 07:19:12.820892 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.820813 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bdtkq"] Sep 15 07:19:12.823449 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.823430 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 15 07:19:12.823727 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.823711 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 15 07:19:12.823815 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.823782 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 15 07:19:12.824050 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.824034 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fpxmd\"" Sep 15 07:19:12.906446 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.906367 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:12.915304 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.915273 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee9505e1-41b8-4f77-b211-c41076a8eac9-config-volume\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:12.915424 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.915309 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee9505e1-41b8-4f77-b211-c41076a8eac9-tmp-dir\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:12.915424 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.915347 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:12.915424 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.915380 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ng8b\" (UniqueName: \"kubernetes.io/projected/ee9505e1-41b8-4f77-b211-c41076a8eac9-kube-api-access-9ng8b\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:12.915424 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.915402 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4169ca54-51ee-4dc7-9280-649b5143d3bf-cert\") pod \"ingress-canary-bdtkq\" (UID: \"4169ca54-51ee-4dc7-9280-649b5143d3bf\") " pod="openshift-ingress-canary/ingress-canary-bdtkq" Sep 15 07:19:12.915581 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:12.915477 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr5pb\" (UniqueName: \"kubernetes.io/projected/4169ca54-51ee-4dc7-9280-649b5143d3bf-kube-api-access-wr5pb\") pod \"ingress-canary-bdtkq\" (UID: \"4169ca54-51ee-4dc7-9280-649b5143d3bf\") " pod="openshift-ingress-canary/ingress-canary-bdtkq" Sep 15 07:19:13.006665 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.006636 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:13.016546 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.016523 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr5pb\" (UniqueName: \"kubernetes.io/projected/4169ca54-51ee-4dc7-9280-649b5143d3bf-kube-api-access-wr5pb\") pod \"ingress-canary-bdtkq\" (UID: \"4169ca54-51ee-4dc7-9280-649b5143d3bf\") " pod="openshift-ingress-canary/ingress-canary-bdtkq" Sep 15 07:19:13.016638 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.016567 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee9505e1-41b8-4f77-b211-c41076a8eac9-config-volume\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:13.016638 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.016583 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee9505e1-41b8-4f77-b211-c41076a8eac9-tmp-dir\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:13.016638 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.016618 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:13.016786 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.016644 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ng8b\" (UniqueName: \"kubernetes.io/projected/ee9505e1-41b8-4f77-b211-c41076a8eac9-kube-api-access-9ng8b\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:13.016786 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.016666 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4169ca54-51ee-4dc7-9280-649b5143d3bf-cert\") pod \"ingress-canary-bdtkq\" (UID: \"4169ca54-51ee-4dc7-9280-649b5143d3bf\") " pod="openshift-ingress-canary/ingress-canary-bdtkq" Sep 15 07:19:13.016902 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:13.016809 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 07:19:13.016902 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:13.016899 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls podName:ee9505e1-41b8-4f77-b211-c41076a8eac9 nodeName:}" failed. No retries permitted until 2026-09-15 07:19:13.516855318 +0000 UTC m=+34.498017478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls") pod "dns-default-bsrr6" (UID: "ee9505e1-41b8-4f77-b211-c41076a8eac9") : secret "dns-default-metrics-tls" not found Sep 15 07:19:13.017044 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.017026 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee9505e1-41b8-4f77-b211-c41076a8eac9-tmp-dir\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:13.017257 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.017241 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ee9505e1-41b8-4f77-b211-c41076a8eac9-config-volume\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:13.020650 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.020634 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4169ca54-51ee-4dc7-9280-649b5143d3bf-cert\") pod \"ingress-canary-bdtkq\" (UID: \"4169ca54-51ee-4dc7-9280-649b5143d3bf\") " pod="openshift-ingress-canary/ingress-canary-bdtkq" Sep 15 07:19:13.025580 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.025561 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr5pb\" (UniqueName: \"kubernetes.io/projected/4169ca54-51ee-4dc7-9280-649b5143d3bf-kube-api-access-wr5pb\") pod \"ingress-canary-bdtkq\" (UID: \"4169ca54-51ee-4dc7-9280-649b5143d3bf\") " pod="openshift-ingress-canary/ingress-canary-bdtkq" Sep 15 07:19:13.026133 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.026114 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ng8b\" (UniqueName: \"kubernetes.io/projected/ee9505e1-41b8-4f77-b211-c41076a8eac9-kube-api-access-9ng8b\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:13.117372 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.117337 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:19:13.117563 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.117411 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:19:13.117563 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:13.117489 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:19:13.117647 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:13.117566 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs podName:f13cc492-490d-478f-b799-6127572cc075 nodeName:}" failed. No retries permitted until 2026-09-15 07:19:45.117546361 +0000 UTC m=+66.098708522 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs") pod "network-metrics-daemon-n9rxl" (UID: "f13cc492-490d-478f-b799-6127572cc075") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 07:19:13.117692 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:13.117661 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 07:19:13.117731 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:13.117710 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret podName:c6f6820e-f789-494a-9a66-fe16e0c2f21a nodeName:}" failed. No retries permitted until 2026-09-15 07:19:45.117698953 +0000 UTC m=+66.098861102 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret") pod "global-pull-secret-syncer-h8b5m" (UID: "c6f6820e-f789-494a-9a66-fe16e0c2f21a") : object "kube-system"/"original-pull-secret" not registered Sep 15 07:19:13.146822 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.146799 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bdtkq" Sep 15 07:19:13.318826 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.318660 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlgn6\" (UniqueName: \"kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6\") pod \"network-check-target-vbjvp\" (UID: \"5ecac077-a8a5-4930-a69c-21402e834160\") " pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:13.318826 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:13.318822 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 07:19:13.319036 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:13.318842 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 07:19:13.319036 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:13.318852 2552 projected.go:196] Error preparing data for projected volume kube-api-access-qlgn6 for pod openshift-network-diagnostics/network-check-target-vbjvp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:19:13.319036 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:13.318926 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6 podName:5ecac077-a8a5-4930-a69c-21402e834160 nodeName:}" failed. No retries permitted until 2026-09-15 07:19:45.318912392 +0000 UTC m=+66.300074550 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlgn6" (UniqueName: "kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6") pod "network-check-target-vbjvp" (UID: "5ecac077-a8a5-4930-a69c-21402e834160") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 07:19:13.456086 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.456015 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bdtkq"] Sep 15 07:19:13.459613 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:19:13.459589 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4169ca54_51ee_4dc7_9280_649b5143d3bf.slice/crio-04c4f66e034b850b6f98a2e9929e1f6834c37c7d9f7131e9b7088c23c7a15643 WatchSource:0}: Error finding container 04c4f66e034b850b6f98a2e9929e1f6834c37c7d9f7131e9b7088c23c7a15643: Status 404 returned error can't find the container with id 04c4f66e034b850b6f98a2e9929e1f6834c37c7d9f7131e9b7088c23c7a15643 Sep 15 07:19:13.520784 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.520753 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:13.521146 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:13.520911 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 07:19:13.521146 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:13.520970 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls podName:ee9505e1-41b8-4f77-b211-c41076a8eac9 nodeName:}" failed. No retries permitted until 2026-09-15 07:19:14.520953273 +0000 UTC m=+35.502115421 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls") pod "dns-default-bsrr6" (UID: "ee9505e1-41b8-4f77-b211-c41076a8eac9") : secret "dns-default-metrics-tls" not found Sep 15 07:19:13.613304 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.613269 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bdtkq" event={"ID":"4169ca54-51ee-4dc7-9280-649b5143d3bf","Type":"ContainerStarted","Data":"04c4f66e034b850b6f98a2e9929e1f6834c37c7d9f7131e9b7088c23c7a15643"} Sep 15 07:19:13.616162 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.616135 2552 generic.go:356] "Generic (PLEG): container finished" podID="b931b03f-d393-4723-b2fe-bc49c3475e67" containerID="ab5feadc662c84000ce2083220930c68e4209640fa9ace5f43a560e1e869d0ef" exitCode=0 Sep 15 07:19:13.616261 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:13.616180 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r2hmp" event={"ID":"b931b03f-d393-4723-b2fe-bc49c3475e67","Type":"ContainerDied","Data":"ab5feadc662c84000ce2083220930c68e4209640fa9ace5f43a560e1e869d0ef"} Sep 15 07:19:14.469107 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.469072 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:19:14.469297 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.469149 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:14.469297 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.469084 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:19:14.472938 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.472918 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 15 07:19:14.473069 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.472952 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ftj64\"" Sep 15 07:19:14.473187 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.473167 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 15 07:19:14.473299 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.473210 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 15 07:19:14.473299 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.473168 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 15 07:19:14.473420 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.473303 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d7gwj\"" Sep 15 07:19:14.530241 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.530202 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:14.530832 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:14.530352 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 07:19:14.530832 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:14.530429 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls podName:ee9505e1-41b8-4f77-b211-c41076a8eac9 nodeName:}" failed. No retries permitted until 2026-09-15 07:19:16.530411609 +0000 UTC m=+37.511573770 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls") pod "dns-default-bsrr6" (UID: "ee9505e1-41b8-4f77-b211-c41076a8eac9") : secret "dns-default-metrics-tls" not found Sep 15 07:19:14.622526 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.622489 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r2hmp" event={"ID":"b931b03f-d393-4723-b2fe-bc49c3475e67","Type":"ContainerStarted","Data":"90f5b2cd17300bb2e0f296a168821150d66c413f2bd19f672d64fb2828116b74"} Sep 15 07:19:14.644335 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.643738 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-r2hmp" podStartSLOduration=10.768512579 podStartE2EDuration="35.643720197s" totalImagesPullingTime="24.875207618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:18:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:14.642512617 +0000 UTC m=+35.623674786" watchObservedRunningTime="2026-09-15 07:19:14.643720197 +0000 UTC m=+35.624882364" Sep 15 07:19:14.763563 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.763530 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:14.824635 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:14.824602 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:16.292048 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:16.292009 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:16.376734 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:16.376705 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:16.544442 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:16.544324 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:16.544442 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:16.544431 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 07:19:16.544584 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:16.544492 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls podName:ee9505e1-41b8-4f77-b211-c41076a8eac9 nodeName:}" failed. No retries permitted until 2026-09-15 07:19:20.544477603 +0000 UTC m=+41.525639765 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls") pod "dns-default-bsrr6" (UID: "ee9505e1-41b8-4f77-b211-c41076a8eac9") : secret "dns-default-metrics-tls" not found Sep 15 07:19:16.628432 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:16.628401 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bdtkq" event={"ID":"4169ca54-51ee-4dc7-9280-649b5143d3bf","Type":"ContainerStarted","Data":"b1f939b9672b8ab6de2e5599b4d7eece5dcdc6a5d9e58d4933acf9f64983f6e2"} Sep 15 07:19:17.614734 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:17.614700 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:17.689639 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:17.689608 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:20.574076 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:20.574037 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:20.574442 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:20.574207 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 07:19:20.574442 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:20.574269 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls podName:ee9505e1-41b8-4f77-b211-c41076a8eac9 nodeName:}" failed. No retries permitted until 2026-09-15 07:19:28.574253124 +0000 UTC m=+49.555415285 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls") pod "dns-default-bsrr6" (UID: "ee9505e1-41b8-4f77-b211-c41076a8eac9") : secret "dns-default-metrics-tls" not found Sep 15 07:19:28.628879 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:28.628835 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:28.629313 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:28.628991 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 07:19:28.629313 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:28.629057 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls podName:ee9505e1-41b8-4f77-b211-c41076a8eac9 nodeName:}" failed. No retries permitted until 2026-09-15 07:19:44.629041104 +0000 UTC m=+65.610203270 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls") pod "dns-default-bsrr6" (UID: "ee9505e1-41b8-4f77-b211-c41076a8eac9") : secret "dns-default-metrics-tls" not found Sep 15 07:19:32.677279 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:32.677251 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:32.751877 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:32.751834 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:33.556937 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.556880 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bdtkq" podStartSLOduration=19.249728708 podStartE2EDuration="21.556844412s" totalImagesPullingTime="2.307115704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:16.643280512 +0000 UTC m=+37.624442681" watchObservedRunningTime="2026-09-15 07:19:33.556844412 +0000 UTC m=+54.538006579" Sep 15 07:19:33.557222 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.557018 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r"] Sep 15 07:19:33.558687 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.558664 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" Sep 15 07:19:33.562035 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.562017 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-696wt\"" Sep 15 07:19:33.563202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.563179 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 15 07:19:33.563304 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.563200 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 15 07:19:33.563304 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.563179 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 15 07:19:33.563304 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.563218 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 15 07:19:33.575637 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.575598 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r"] Sep 15 07:19:33.665860 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.665823 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/049cc39c-70fa-4a11-a24b-05f87cd9c1b6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-554bf7566d-vxg4r\" (UID: \"049cc39c-70fa-4a11-a24b-05f87cd9c1b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" Sep 15 07:19:33.666060 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.665939 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xzvk\" (UniqueName: \"kubernetes.io/projected/049cc39c-70fa-4a11-a24b-05f87cd9c1b6-kube-api-access-2xzvk\") pod \"managed-serviceaccount-addon-agent-554bf7566d-vxg4r\" (UID: \"049cc39c-70fa-4a11-a24b-05f87cd9c1b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" Sep 15 07:19:33.767207 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.767172 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xzvk\" (UniqueName: \"kubernetes.io/projected/049cc39c-70fa-4a11-a24b-05f87cd9c1b6-kube-api-access-2xzvk\") pod \"managed-serviceaccount-addon-agent-554bf7566d-vxg4r\" (UID: \"049cc39c-70fa-4a11-a24b-05f87cd9c1b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" Sep 15 07:19:33.767677 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.767239 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/049cc39c-70fa-4a11-a24b-05f87cd9c1b6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-554bf7566d-vxg4r\" (UID: \"049cc39c-70fa-4a11-a24b-05f87cd9c1b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" Sep 15 07:19:33.770625 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.770599 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/049cc39c-70fa-4a11-a24b-05f87cd9c1b6-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-554bf7566d-vxg4r\" (UID: \"049cc39c-70fa-4a11-a24b-05f87cd9c1b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" Sep 15 07:19:33.777083 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.777061 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xzvk\" (UniqueName: \"kubernetes.io/projected/049cc39c-70fa-4a11-a24b-05f87cd9c1b6-kube-api-access-2xzvk\") pod \"managed-serviceaccount-addon-agent-554bf7566d-vxg4r\" (UID: \"049cc39c-70fa-4a11-a24b-05f87cd9c1b6\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" Sep 15 07:19:33.879547 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:33.879469 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" Sep 15 07:19:34.148688 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:34.148618 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r"] Sep 15 07:19:34.152359 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:19:34.152331 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod049cc39c_70fa_4a11_a24b_05f87cd9c1b6.slice/crio-fb8ba95f7b61081e40095d927cdc7b5b643b1be02394466c0da6e0c5419340d1 WatchSource:0}: Error finding container fb8ba95f7b61081e40095d927cdc7b5b643b1be02394466c0da6e0c5419340d1: Status 404 returned error can't find the container with id fb8ba95f7b61081e40095d927cdc7b5b643b1be02394466c0da6e0c5419340d1 Sep 15 07:19:34.670974 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:34.670937 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" event={"ID":"049cc39c-70fa-4a11-a24b-05f87cd9c1b6","Type":"ContainerStarted","Data":"fb8ba95f7b61081e40095d927cdc7b5b643b1be02394466c0da6e0c5419340d1"} Sep 15 07:19:37.680230 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:37.680194 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" event={"ID":"049cc39c-70fa-4a11-a24b-05f87cd9c1b6","Type":"ContainerStarted","Data":"b2be33b52cebcacd6062eac1152e5d5e844d7425af07727d79df678c5ec6cc4f"} Sep 15 07:19:37.698410 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:37.698369 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" podStartSLOduration=1.665337361 podStartE2EDuration="4.698352769s" totalImagesPullingTime="3.033015408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:19:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:37.697401835 +0000 UTC m=+58.678564003" watchObservedRunningTime="2026-09-15 07:19:37.698352769 +0000 UTC m=+58.679514936" Sep 15 07:19:38.606904 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:38.606853 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkh69" Sep 15 07:19:38.734733 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:38.734698 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:38.796260 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:38.796221 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:41.087582 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:41.087547 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:19:44.646986 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:44.646945 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:19:44.647387 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:44.647110 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 07:19:44.647387 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:44.647171 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls podName:ee9505e1-41b8-4f77-b211-c41076a8eac9 nodeName:}" failed. No retries permitted until 2026-09-15 07:20:16.647155858 +0000 UTC m=+97.628318020 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls") pod "dns-default-bsrr6" (UID: "ee9505e1-41b8-4f77-b211-c41076a8eac9") : secret "dns-default-metrics-tls" not found Sep 15 07:19:45.150398 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.150349 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:19:45.150561 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.150435 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:19:45.153261 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.153245 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 15 07:19:45.153261 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.153252 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 15 07:19:45.161398 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:45.161381 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 15 07:19:45.161466 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:19:45.161457 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs podName:f13cc492-490d-478f-b799-6127572cc075 nodeName:}" failed. No retries permitted until 2026-09-15 07:20:49.16144163 +0000 UTC m=+130.142603779 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs") pod "network-metrics-daemon-n9rxl" (UID: "f13cc492-490d-478f-b799-6127572cc075") : secret "metrics-daemon-secret" not found Sep 15 07:19:45.163474 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.163457 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c6f6820e-f789-494a-9a66-fe16e0c2f21a-original-pull-secret\") pod \"global-pull-secret-syncer-h8b5m\" (UID: \"c6f6820e-f789-494a-9a66-fe16e0c2f21a\") " pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:19:45.352233 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.352192 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlgn6\" (UniqueName: \"kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6\") pod \"network-check-target-vbjvp\" (UID: \"5ecac077-a8a5-4930-a69c-21402e834160\") " pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:45.355445 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.355428 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 15 07:19:45.365230 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.365216 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 15 07:19:45.375338 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.375311 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlgn6\" (UniqueName: \"kubernetes.io/projected/5ecac077-a8a5-4930-a69c-21402e834160-kube-api-access-qlgn6\") pod \"network-check-target-vbjvp\" (UID: \"5ecac077-a8a5-4930-a69c-21402e834160\") " pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:45.381084 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.381062 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h8b5m" Sep 15 07:19:45.398236 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.398217 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d7gwj\"" Sep 15 07:19:45.406343 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.406288 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:45.649821 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.649789 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h8b5m"] Sep 15 07:19:45.652447 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:19:45.652420 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f6820e_f789_494a_9a66_fe16e0c2f21a.slice/crio-cc0ccdadd0fb9e241ad24eedec6e5ab4b12c0cc60dad0195c47e847d16b70cf1 WatchSource:0}: Error finding container cc0ccdadd0fb9e241ad24eedec6e5ab4b12c0cc60dad0195c47e847d16b70cf1: Status 404 returned error can't find the container with id cc0ccdadd0fb9e241ad24eedec6e5ab4b12c0cc60dad0195c47e847d16b70cf1 Sep 15 07:19:45.682939 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.682912 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vbjvp"] Sep 15 07:19:45.686002 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:19:45.685980 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ecac077_a8a5_4930_a69c_21402e834160.slice/crio-7601d3ab603809f7fab3c0b1e3bed7157dbbdf066724d33a310acf7c79637dc1 WatchSource:0}: Error finding container 7601d3ab603809f7fab3c0b1e3bed7157dbbdf066724d33a310acf7c79637dc1: Status 404 returned error can't find the container with id 7601d3ab603809f7fab3c0b1e3bed7157dbbdf066724d33a310acf7c79637dc1 Sep 15 07:19:45.700240 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.700215 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vbjvp" event={"ID":"5ecac077-a8a5-4930-a69c-21402e834160","Type":"ContainerStarted","Data":"7601d3ab603809f7fab3c0b1e3bed7157dbbdf066724d33a310acf7c79637dc1"} Sep 15 07:19:45.701373 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:45.701346 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h8b5m" event={"ID":"c6f6820e-f789-494a-9a66-fe16e0c2f21a","Type":"ContainerStarted","Data":"cc0ccdadd0fb9e241ad24eedec6e5ab4b12c0cc60dad0195c47e847d16b70cf1"} Sep 15 07:19:50.715769 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:50.715732 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vbjvp" event={"ID":"5ecac077-a8a5-4930-a69c-21402e834160","Type":"ContainerStarted","Data":"baf3f066c3115a8226622eefc5b9bc45133b33169602768d9c5d49251a5d496d"} Sep 15 07:19:50.716116 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:50.715900 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:19:50.717377 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:50.717348 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h8b5m" event={"ID":"c6f6820e-f789-494a-9a66-fe16e0c2f21a","Type":"ContainerStarted","Data":"9e7ab4f06c35ca97bd03becb76490e361114744dbfa5f7106dd9c998d5cea30b"} Sep 15 07:19:50.731554 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:50.731510 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vbjvp" podStartSLOduration=66.795235147 podStartE2EDuration="1m11.731495155s" totalImagesPullingTime="4.936260008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:50.730644032 +0000 UTC m=+71.711806200" watchObservedRunningTime="2026-09-15 07:19:50.731495155 +0000 UTC m=+71.712657322" Sep 15 07:19:50.746056 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:19:50.746005 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h8b5m" podStartSLOduration=66.773699668 podStartE2EDuration="1m11.745985269s" totalImagesPullingTime="4.972285601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:19:50.745176806 +0000 UTC m=+71.726338973" watchObservedRunningTime="2026-09-15 07:19:50.745985269 +0000 UTC m=+71.727147436" Sep 15 07:20:16.664894 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:20:16.664850 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:20:16.665238 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:20:16.664986 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 07:20:16.665238 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:20:16.665046 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls podName:ee9505e1-41b8-4f77-b211-c41076a8eac9 nodeName:}" failed. No retries permitted until 2026-09-15 07:21:20.665032439 +0000 UTC m=+161.646194583 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls") pod "dns-default-bsrr6" (UID: "ee9505e1-41b8-4f77-b211-c41076a8eac9") : secret "dns-default-metrics-tls" not found Sep 15 07:20:21.722745 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:20:21.722714 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vbjvp" Sep 15 07:20:49.172506 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:20:49.172463 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:20:49.173036 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:20:49.172591 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 15 07:20:49.173036 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:20:49.172680 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs podName:f13cc492-490d-478f-b799-6127572cc075 nodeName:}" failed. No retries permitted until 2026-09-15 07:22:51.172665879 +0000 UTC m=+252.153828027 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs") pod "network-metrics-daemon-n9rxl" (UID: "f13cc492-490d-478f-b799-6127572cc075") : secret "metrics-daemon-secret" not found Sep 15 07:21:15.842437 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:21:15.842393 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bsrr6" podUID="ee9505e1-41b8-4f77-b211-c41076a8eac9" Sep 15 07:21:15.921577 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:21:15.921546 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bsrr6" Sep 15 07:21:17.488980 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:21:17.488945 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-n9rxl" podUID="f13cc492-490d-478f-b799-6127572cc075" Sep 15 07:21:20.684193 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:21:20.684158 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:21:20.684659 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:21:20.684284 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 07:21:20.684659 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:21:20.684342 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls podName:ee9505e1-41b8-4f77-b211-c41076a8eac9 nodeName:}" failed. No retries permitted until 2026-09-15 07:23:22.684328327 +0000 UTC m=+283.665490471 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls") pod "dns-default-bsrr6" (UID: "ee9505e1-41b8-4f77-b211-c41076a8eac9") : secret "dns-default-metrics-tls" not found Sep 15 07:21:30.468365 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:21:30.468330 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:21:37.974998 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:21:37.974964 2552 generic.go:356] "Generic (PLEG): container finished" podID="049cc39c-70fa-4a11-a24b-05f87cd9c1b6" containerID="b2be33b52cebcacd6062eac1152e5d5e844d7425af07727d79df678c5ec6cc4f" exitCode=255 Sep 15 07:21:37.975481 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:21:37.975016 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" event={"ID":"049cc39c-70fa-4a11-a24b-05f87cd9c1b6","Type":"ContainerDied","Data":"b2be33b52cebcacd6062eac1152e5d5e844d7425af07727d79df678c5ec6cc4f"} Sep 15 07:21:37.980558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:21:37.980539 2552 scope.go:117] "RemoveContainer" containerID="b2be33b52cebcacd6062eac1152e5d5e844d7425af07727d79df678c5ec6cc4f" Sep 15 07:21:38.981877 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:21:38.981829 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554bf7566d-vxg4r" event={"ID":"049cc39c-70fa-4a11-a24b-05f87cd9c1b6","Type":"ContainerStarted","Data":"31abeac71618954c60975d9ecf6ce0c204ae63714eaa43afe4b694adb318e00e"} Sep 15 07:21:58.235891 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:21:58.235847 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:26.987614 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.987580 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-c5cb56bb8-jq29f"] Sep 15 07:22:26.989678 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.989661 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b6ff5f5df-rzw29"] Sep 15 07:22:26.989825 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.989799 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:26.991567 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.991547 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:26.992015 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.991995 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 15 07:22:26.992567 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.992550 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 15 07:22:26.992781 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.992767 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 15 07:22:26.992838 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.992802 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 15 07:22:26.992952 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.992807 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8qj4j\"" Sep 15 07:22:26.993225 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.993209 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 15 07:22:26.993225 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.993222 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 15 07:22:26.994063 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.994047 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 15 07:22:26.994315 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.994301 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 15 07:22:26.995767 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.995747 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rxml2\"" Sep 15 07:22:26.995850 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:26.995774 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 15 07:22:27.004829 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.004810 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 15 07:22:27.010409 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.010391 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c5cb56bb8-jq29f"] Sep 15 07:22:27.012579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.012559 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b6ff5f5df-rzw29"] Sep 15 07:22:27.081728 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.081696 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-46pvm"] Sep 15 07:22:27.083473 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.083458 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-cdwxq"] Sep 15 07:22:27.083598 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.083581 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.085532 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.085509 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.086542 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.086521 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 15 07:22:27.086635 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.086619 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 15 07:22:27.086772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.086757 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-jrthd\"" Sep 15 07:22:27.089133 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.089118 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 15 07:22:27.089790 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.089778 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 15 07:22:27.091694 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.091670 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 15 07:22:27.092350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.092321 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 15 07:22:27.093727 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.093482 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 15 07:22:27.093881 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.093846 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ckdvm\"" Sep 15 07:22:27.096366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.094571 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 15 07:22:27.099603 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.099586 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 15 07:22:27.100341 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.100325 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 15 07:22:27.104311 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.104291 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-46pvm"] Sep 15 07:22:27.105612 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.105592 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-cdwxq"] Sep 15 07:22:27.117074 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117050 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/449e530a-1a2b-4707-8272-48ec5851253d-snapshots\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.117173 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117077 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-stats-auth\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.117173 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117095 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/449e530a-1a2b-4707-8272-48ec5851253d-tmp\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.117173 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117110 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/449e530a-1a2b-4707-8272-48ec5851253d-trusted-ca-bundle\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.117173 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117125 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/449e530a-1a2b-4707-8272-48ec5851253d-service-ca-bundle\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.117366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117178 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2zfb\" (UniqueName: \"kubernetes.io/projected/449e530a-1a2b-4707-8272-48ec5851253d-kube-api-access-d2zfb\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.117366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117204 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-certificates\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.117366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117224 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/449e530a-1a2b-4707-8272-48ec5851253d-serving-cert\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.117366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117274 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-image-registry-private-configuration\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.117366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117316 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.117366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117351 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7ss6\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-kube-api-access-r7ss6\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.117659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117376 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l7sc\" (UniqueName: \"kubernetes.io/projected/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-kube-api-access-7l7sc\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.117659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117407 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-trusted-ca\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.117659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117445 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-default-certificate\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.117659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117468 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-ca-trust-extracted\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.117659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117490 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-bound-sa-token\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.117659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117517 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a0dd4f12-537f-42ae-99c6-2d900b956860-serving-cert\") pod \"console-operator-696884fb9c-cdwxq\" (UID: \"a0dd4f12-537f-42ae-99c6-2d900b956860\") " pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.117659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117542 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.117659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117583 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-installation-pull-secrets\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.117659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117631 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.117659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117653 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0dd4f12-537f-42ae-99c6-2d900b956860-trusted-ca\") pod \"console-operator-696884fb9c-cdwxq\" (UID: \"a0dd4f12-537f-42ae-99c6-2d900b956860\") " pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.118058 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117683 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a0dd4f12-537f-42ae-99c6-2d900b956860-config\") pod \"console-operator-696884fb9c-cdwxq\" (UID: \"a0dd4f12-537f-42ae-99c6-2d900b956860\") " pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.118058 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.117719 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n59m\" (UniqueName: \"kubernetes.io/projected/a0dd4f12-537f-42ae-99c6-2d900b956860-kube-api-access-6n59m\") pod \"console-operator-696884fb9c-cdwxq\" (UID: \"a0dd4f12-537f-42ae-99c6-2d900b956860\") " pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.218336 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218300 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/449e530a-1a2b-4707-8272-48ec5851253d-snapshots\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.218336 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218335 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-stats-auth\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.218583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218350 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/449e530a-1a2b-4707-8272-48ec5851253d-tmp\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.218583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218378 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/449e530a-1a2b-4707-8272-48ec5851253d-trusted-ca-bundle\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.218583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218394 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/449e530a-1a2b-4707-8272-48ec5851253d-service-ca-bundle\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.218583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218443 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2zfb\" (UniqueName: \"kubernetes.io/projected/449e530a-1a2b-4707-8272-48ec5851253d-kube-api-access-d2zfb\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.218583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218478 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-certificates\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.218583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218504 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/449e530a-1a2b-4707-8272-48ec5851253d-serving-cert\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.218583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218536 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-image-registry-private-configuration\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.218949 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218820 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.218949 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218859 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7ss6\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-kube-api-access-r7ss6\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.218949 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218926 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l7sc\" (UniqueName: \"kubernetes.io/projected/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-kube-api-access-7l7sc\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.219108 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218952 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-trusted-ca\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.219108 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.218990 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-default-certificate\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.219108 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219004 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/449e530a-1a2b-4707-8272-48ec5851253d-tmp\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.219108 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219014 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-ca-trust-extracted\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.219108 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219037 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-bound-sa-token\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.219108 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219068 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a0dd4f12-537f-42ae-99c6-2d900b956860-serving-cert\") pod \"console-operator-696884fb9c-cdwxq\" (UID: \"a0dd4f12-537f-42ae-99c6-2d900b956860\") " pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.219108 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219106 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.219454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219113 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/449e530a-1a2b-4707-8272-48ec5851253d-snapshots\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.219454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219153 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-installation-pull-secrets\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.219454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219168 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-certificates\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.219454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219231 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.219454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219263 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0dd4f12-537f-42ae-99c6-2d900b956860-trusted-ca\") pod \"console-operator-696884fb9c-cdwxq\" (UID: \"a0dd4f12-537f-42ae-99c6-2d900b956860\") " pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.219454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219303 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a0dd4f12-537f-42ae-99c6-2d900b956860-config\") pod \"console-operator-696884fb9c-cdwxq\" (UID: \"a0dd4f12-537f-42ae-99c6-2d900b956860\") " pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.219454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219337 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n59m\" (UniqueName: \"kubernetes.io/projected/a0dd4f12-537f-42ae-99c6-2d900b956860-kube-api-access-6n59m\") pod \"console-operator-696884fb9c-cdwxq\" (UID: \"a0dd4f12-537f-42ae-99c6-2d900b956860\") " pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.219454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219353 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/449e530a-1a2b-4707-8272-48ec5851253d-trusted-ca-bundle\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.219910 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:27.219494 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle podName:a951ccbf-4363-44e1-9e9b-45a3ae082e7e nodeName:}" failed. No retries permitted until 2026-09-15 07:22:27.719467699 +0000 UTC m=+228.700629863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle") pod "router-default-c5cb56bb8-jq29f" (UID: "a951ccbf-4363-44e1-9e9b-45a3ae082e7e") : configmap references non-existent config key: service-ca.crt Sep 15 07:22:27.219910 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.219783 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/449e530a-1a2b-4707-8272-48ec5851253d-service-ca-bundle\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.220751 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.220083 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-trusted-ca\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.220751 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.220093 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a0dd4f12-537f-42ae-99c6-2d900b956860-config\") pod \"console-operator-696884fb9c-cdwxq\" (UID: \"a0dd4f12-537f-42ae-99c6-2d900b956860\") " pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.220751 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:27.220169 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 15 07:22:27.220751 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:27.220229 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs podName:a951ccbf-4363-44e1-9e9b-45a3ae082e7e nodeName:}" failed. No retries permitted until 2026-09-15 07:22:27.720211644 +0000 UTC m=+228.701373807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs") pod "router-default-c5cb56bb8-jq29f" (UID: "a951ccbf-4363-44e1-9e9b-45a3ae082e7e") : secret "router-metrics-certs-default" not found Sep 15 07:22:27.220751 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.220390 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-ca-trust-extracted\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.220751 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.220646 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a0dd4f12-537f-42ae-99c6-2d900b956860-trusted-ca\") pod \"console-operator-696884fb9c-cdwxq\" (UID: \"a0dd4f12-537f-42ae-99c6-2d900b956860\") " pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.220751 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:27.220670 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 15 07:22:27.220751 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:27.220687 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b6ff5f5df-rzw29: secret "image-registry-tls" not found Sep 15 07:22:27.220751 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:27.220728 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls podName:98bd66da-f44c-42dd-b8c9-a502aa7a86c5 nodeName:}" failed. No retries permitted until 2026-09-15 07:22:27.720714489 +0000 UTC m=+228.701876648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls") pod "image-registry-b6ff5f5df-rzw29" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5") : secret "image-registry-tls" not found Sep 15 07:22:27.221648 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.221625 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-image-registry-private-configuration\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.221821 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.221798 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-installation-pull-secrets\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.221924 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.221848 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-stats-auth\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.222204 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.222179 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/449e530a-1a2b-4707-8272-48ec5851253d-serving-cert\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.222547 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.222531 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-default-certificate\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.222682 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.222667 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a0dd4f12-537f-42ae-99c6-2d900b956860-serving-cert\") pod \"console-operator-696884fb9c-cdwxq\" (UID: \"a0dd4f12-537f-42ae-99c6-2d900b956860\") " pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.226713 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.226692 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2zfb\" (UniqueName: \"kubernetes.io/projected/449e530a-1a2b-4707-8272-48ec5851253d-kube-api-access-d2zfb\") pod \"insights-operator-5844b856db-46pvm\" (UID: \"449e530a-1a2b-4707-8272-48ec5851253d\") " pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.228498 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.228474 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n59m\" (UniqueName: \"kubernetes.io/projected/a0dd4f12-537f-42ae-99c6-2d900b956860-kube-api-access-6n59m\") pod \"console-operator-696884fb9c-cdwxq\" (UID: \"a0dd4f12-537f-42ae-99c6-2d900b956860\") " pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.228647 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.228592 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-bound-sa-token\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.229881 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.229850 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l7sc\" (UniqueName: \"kubernetes.io/projected/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-kube-api-access-7l7sc\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.230009 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.229993 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7ss6\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-kube-api-access-r7ss6\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.240706 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.240649 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:27.337165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.337136 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:27.394086 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.394058 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-46pvm" Sep 15 07:22:27.400673 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.400652 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:27.668842 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.668818 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-46pvm"] Sep 15 07:22:27.671170 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:22:27.671141 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod449e530a_1a2b_4707_8272_48ec5851253d.slice/crio-8da6f223fc273723a005ef7585861d7691d8b3cf4a9db01a55ba2ced6865933f WatchSource:0}: Error finding container 8da6f223fc273723a005ef7585861d7691d8b3cf4a9db01a55ba2ced6865933f: Status 404 returned error can't find the container with id 8da6f223fc273723a005ef7585861d7691d8b3cf4a9db01a55ba2ced6865933f Sep 15 07:22:27.674299 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.674279 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-cdwxq"] Sep 15 07:22:27.677241 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:22:27.677215 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0dd4f12_537f_42ae_99c6_2d900b956860.slice/crio-8f249d15ebf552d054fcb38c5e74babedfef7cbb5b2a209737b59a67974406db WatchSource:0}: Error finding container 8f249d15ebf552d054fcb38c5e74babedfef7cbb5b2a209737b59a67974406db: Status 404 returned error can't find the container with id 8f249d15ebf552d054fcb38c5e74babedfef7cbb5b2a209737b59a67974406db Sep 15 07:22:27.723530 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.723497 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.723687 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.723548 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:27.723687 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:27.723648 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 15 07:22:27.723687 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:27.723649 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 15 07:22:27.723687 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:27.723672 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:27.723834 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:27.723659 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b6ff5f5df-rzw29: secret "image-registry-tls" not found Sep 15 07:22:27.723834 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:27.723716 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs podName:a951ccbf-4363-44e1-9e9b-45a3ae082e7e nodeName:}" failed. No retries permitted until 2026-09-15 07:22:28.723695432 +0000 UTC m=+229.704857579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs") pod "router-default-c5cb56bb8-jq29f" (UID: "a951ccbf-4363-44e1-9e9b-45a3ae082e7e") : secret "router-metrics-certs-default" not found Sep 15 07:22:27.723834 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:27.723748 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle podName:a951ccbf-4363-44e1-9e9b-45a3ae082e7e nodeName:}" failed. No retries permitted until 2026-09-15 07:22:28.723736897 +0000 UTC m=+229.704899071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle") pod "router-default-c5cb56bb8-jq29f" (UID: "a951ccbf-4363-44e1-9e9b-45a3ae082e7e") : configmap references non-existent config key: service-ca.crt Sep 15 07:22:27.723834 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:27.723761 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls podName:98bd66da-f44c-42dd-b8c9-a502aa7a86c5 nodeName:}" failed. No retries permitted until 2026-09-15 07:22:28.723755215 +0000 UTC m=+229.704917360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls") pod "image-registry-b6ff5f5df-rzw29" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5") : secret "image-registry-tls" not found Sep 15 07:22:28.123165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:28.123130 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" event={"ID":"a0dd4f12-537f-42ae-99c6-2d900b956860","Type":"ContainerStarted","Data":"8f249d15ebf552d054fcb38c5e74babedfef7cbb5b2a209737b59a67974406db"} Sep 15 07:22:28.124126 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:28.124104 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-46pvm" event={"ID":"449e530a-1a2b-4707-8272-48ec5851253d","Type":"ContainerStarted","Data":"8da6f223fc273723a005ef7585861d7691d8b3cf4a9db01a55ba2ced6865933f"} Sep 15 07:22:28.731877 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:28.731825 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:28.732087 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:28.731903 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:28.732087 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:28.731962 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:28.732087 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:28.732007 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 15 07:22:28.732087 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:28.732063 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 15 07:22:28.732087 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:28.732082 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b6ff5f5df-rzw29: secret "image-registry-tls" not found Sep 15 07:22:28.732288 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:28.732084 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs podName:a951ccbf-4363-44e1-9e9b-45a3ae082e7e nodeName:}" failed. No retries permitted until 2026-09-15 07:22:30.732066988 +0000 UTC m=+231.713229152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs") pod "router-default-c5cb56bb8-jq29f" (UID: "a951ccbf-4363-44e1-9e9b-45a3ae082e7e") : secret "router-metrics-certs-default" not found Sep 15 07:22:28.732288 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:28.732146 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle podName:a951ccbf-4363-44e1-9e9b-45a3ae082e7e nodeName:}" failed. No retries permitted until 2026-09-15 07:22:30.73212684 +0000 UTC m=+231.713288985 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle") pod "router-default-c5cb56bb8-jq29f" (UID: "a951ccbf-4363-44e1-9e9b-45a3ae082e7e") : configmap references non-existent config key: service-ca.crt Sep 15 07:22:28.732288 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:28.732167 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls podName:98bd66da-f44c-42dd-b8c9-a502aa7a86c5 nodeName:}" failed. No retries permitted until 2026-09-15 07:22:30.732157848 +0000 UTC m=+231.713319999 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls") pod "image-registry-b6ff5f5df-rzw29" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5") : secret "image-registry-tls" not found Sep 15 07:22:30.751035 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:30.750992 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:30.751492 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:30.751067 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:30.751492 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:30.751112 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:30.751492 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:30.751169 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle podName:a951ccbf-4363-44e1-9e9b-45a3ae082e7e nodeName:}" failed. No retries permitted until 2026-09-15 07:22:34.751146702 +0000 UTC m=+235.732308854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle") pod "router-default-c5cb56bb8-jq29f" (UID: "a951ccbf-4363-44e1-9e9b-45a3ae082e7e") : configmap references non-existent config key: service-ca.crt Sep 15 07:22:30.751492 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:30.751225 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 15 07:22:30.751492 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:30.751274 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs podName:a951ccbf-4363-44e1-9e9b-45a3ae082e7e nodeName:}" failed. No retries permitted until 2026-09-15 07:22:34.751262643 +0000 UTC m=+235.732424804 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs") pod "router-default-c5cb56bb8-jq29f" (UID: "a951ccbf-4363-44e1-9e9b-45a3ae082e7e") : secret "router-metrics-certs-default" not found Sep 15 07:22:30.751492 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:30.751232 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 15 07:22:30.751492 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:30.751293 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b6ff5f5df-rzw29: secret "image-registry-tls" not found Sep 15 07:22:30.751492 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:30.751314 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls podName:98bd66da-f44c-42dd-b8c9-a502aa7a86c5 nodeName:}" failed. No retries permitted until 2026-09-15 07:22:34.751308526 +0000 UTC m=+235.732470671 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls") pod "image-registry-b6ff5f5df-rzw29" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5") : secret "image-registry-tls" not found Sep 15 07:22:31.136006 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.135927 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/0.log" Sep 15 07:22:31.136006 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.135970 2552 generic.go:356] "Generic (PLEG): container finished" podID="a0dd4f12-537f-42ae-99c6-2d900b956860" containerID="58beef32a5ab0926655fe83a824fd38217d619c071fd57db6957d19e4727ff15" exitCode=255 Sep 15 07:22:31.136211 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.136004 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" event={"ID":"a0dd4f12-537f-42ae-99c6-2d900b956860","Type":"ContainerDied","Data":"58beef32a5ab0926655fe83a824fd38217d619c071fd57db6957d19e4727ff15"} Sep 15 07:22:31.136341 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.136316 2552 scope.go:117] "RemoveContainer" containerID="58beef32a5ab0926655fe83a824fd38217d619c071fd57db6957d19e4727ff15" Sep 15 07:22:31.137624 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.137593 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-46pvm" event={"ID":"449e530a-1a2b-4707-8272-48ec5851253d","Type":"ContainerStarted","Data":"29ab3197ded1e683ea410b60a4d10dc0d01ae4e4f1549c87d11af6db37f59d33"} Sep 15 07:22:31.167299 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.167248 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-46pvm" podStartSLOduration=1.662318098 podStartE2EDuration="4.167232122s" totalImagesPullingTime="2.504914024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:22:31.166559945 +0000 UTC m=+232.147722113" watchObservedRunningTime="2026-09-15 07:22:31.167232122 +0000 UTC m=+232.148394288" Sep 15 07:22:31.417236 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.417158 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:31.420364 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.420341 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cttzt"] Sep 15 07:22:31.423439 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.423422 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.426660 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.426640 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 15 07:22:31.426758 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.426641 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 15 07:22:31.426758 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.426710 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mn7st\"" Sep 15 07:22:31.433154 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.433134 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cttzt"] Sep 15 07:22:31.482088 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.482056 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:31.557790 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.557756 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2pzp\" (UniqueName: \"kubernetes.io/projected/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-api-access-d2pzp\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.557991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.557803 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.557991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.557888 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-crio-socket\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.557991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.557967 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.558155 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.558033 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-data-volume\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.658784 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.658747 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.658948 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.658796 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-data-volume\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.658948 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:31.658852 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 07:22:31.658948 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:31.658939 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm podName:5ef49b13-2791-4cf4-9de5-8ac40eae2eee nodeName:}" failed. No retries permitted until 2026-09-15 07:22:32.158920821 +0000 UTC m=+233.140082968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cttzt" (UID: "5ef49b13-2791-4cf4-9de5-8ac40eae2eee") : configmap "kube-rbac-proxy" not found Sep 15 07:22:31.659059 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.658968 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2pzp\" (UniqueName: \"kubernetes.io/projected/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-api-access-d2pzp\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.659059 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.659012 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.659159 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.659075 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-crio-socket\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.659159 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.659101 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-data-volume\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.659244 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:31.659167 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 07:22:31.659244 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.659170 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-crio-socket\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.659244 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:31.659226 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls podName:5ef49b13-2791-4cf4-9de5-8ac40eae2eee nodeName:}" failed. No retries permitted until 2026-09-15 07:22:32.159203092 +0000 UTC m=+233.140365237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cttzt" (UID: "5ef49b13-2791-4cf4-9de5-8ac40eae2eee") : secret "insights-runtime-extractor-tls" not found Sep 15 07:22:31.666816 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.666794 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2pzp\" (UniqueName: \"kubernetes.io/projected/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-api-access-d2pzp\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:31.788955 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:31.788921 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:32.142099 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:32.142024 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/1.log" Sep 15 07:22:32.142460 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:32.142441 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/0.log" Sep 15 07:22:32.142521 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:32.142478 2552 generic.go:356] "Generic (PLEG): container finished" podID="a0dd4f12-537f-42ae-99c6-2d900b956860" containerID="88c16263b82beed42bc8848e1e0070d6d589bc75e8cd9347e8d81d9b7ffa88d8" exitCode=255 Sep 15 07:22:32.142559 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:32.142546 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" event={"ID":"a0dd4f12-537f-42ae-99c6-2d900b956860","Type":"ContainerDied","Data":"88c16263b82beed42bc8848e1e0070d6d589bc75e8cd9347e8d81d9b7ffa88d8"} Sep 15 07:22:32.142610 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:32.142593 2552 scope.go:117] "RemoveContainer" containerID="58beef32a5ab0926655fe83a824fd38217d619c071fd57db6957d19e4727ff15" Sep 15 07:22:32.142819 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:32.142798 2552 scope.go:117] "RemoveContainer" containerID="88c16263b82beed42bc8848e1e0070d6d589bc75e8cd9347e8d81d9b7ffa88d8" Sep 15 07:22:32.143042 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:32.143023 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-cdwxq_openshift-console-operator(a0dd4f12-537f-42ae-99c6-2d900b956860)\"" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" podUID="a0dd4f12-537f-42ae-99c6-2d900b956860" Sep 15 07:22:32.164141 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:32.164112 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:32.164265 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:32.164227 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:32.164354 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:32.164338 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 07:22:32.164409 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:32.164358 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 07:22:32.164479 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:32.164410 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm podName:5ef49b13-2791-4cf4-9de5-8ac40eae2eee nodeName:}" failed. No retries permitted until 2026-09-15 07:22:33.164384177 +0000 UTC m=+234.145546342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cttzt" (UID: "5ef49b13-2791-4cf4-9de5-8ac40eae2eee") : configmap "kube-rbac-proxy" not found Sep 15 07:22:32.164543 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:32.164493 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls podName:5ef49b13-2791-4cf4-9de5-8ac40eae2eee nodeName:}" failed. No retries permitted until 2026-09-15 07:22:33.164481036 +0000 UTC m=+234.145643180 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cttzt" (UID: "5ef49b13-2791-4cf4-9de5-8ac40eae2eee") : secret "insights-runtime-extractor-tls" not found Sep 15 07:22:32.248879 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:32.248805 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:32.310671 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:32.310644 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:32.784385 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:32.784353 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:33.147284 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:33.147197 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/1.log" Sep 15 07:22:33.147738 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:33.147622 2552 scope.go:117] "RemoveContainer" containerID="88c16263b82beed42bc8848e1e0070d6d589bc75e8cd9347e8d81d9b7ffa88d8" Sep 15 07:22:33.147831 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:33.147809 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-cdwxq_openshift-console-operator(a0dd4f12-537f-42ae-99c6-2d900b956860)\"" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" podUID="a0dd4f12-537f-42ae-99c6-2d900b956860" Sep 15 07:22:33.173250 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:33.173217 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:33.173394 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:33.173277 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:33.173394 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:33.173347 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 07:22:33.173394 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:33.173358 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 07:22:33.173497 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:33.173397 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm podName:5ef49b13-2791-4cf4-9de5-8ac40eae2eee nodeName:}" failed. No retries permitted until 2026-09-15 07:22:35.173382742 +0000 UTC m=+236.154544887 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cttzt" (UID: "5ef49b13-2791-4cf4-9de5-8ac40eae2eee") : configmap "kube-rbac-proxy" not found Sep 15 07:22:33.173497 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:33.173409 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls podName:5ef49b13-2791-4cf4-9de5-8ac40eae2eee nodeName:}" failed. No retries permitted until 2026-09-15 07:22:35.173403313 +0000 UTC m=+236.154565458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cttzt" (UID: "5ef49b13-2791-4cf4-9de5-8ac40eae2eee") : secret "insights-runtime-extractor-tls" not found Sep 15 07:22:33.231017 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:33.230984 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:33.332936 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:33.332901 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:33.370101 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:33.370079 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xbtjp_42649c54-356b-4265-9d26-2c42d9b0f265/dns-node-resolver/0.log" Sep 15 07:22:33.486441 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:33.486408 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:33.588277 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:33.588236 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:33.790736 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:33.790650 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:33.889728 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:33.889701 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:34.570007 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:34.569983 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-njnqn_7e2134f2-9ba8-432a-9ba9-aa8e313a86c6/node-ca/0.log" Sep 15 07:22:34.785547 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:34.785511 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:34.785725 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:34.785592 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:34.785725 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:34.785639 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:34.785725 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:34.785665 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 15 07:22:34.785892 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:34.785729 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle podName:a951ccbf-4363-44e1-9e9b-45a3ae082e7e nodeName:}" failed. No retries permitted until 2026-09-15 07:22:42.78571601 +0000 UTC m=+243.766878174 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle") pod "router-default-c5cb56bb8-jq29f" (UID: "a951ccbf-4363-44e1-9e9b-45a3ae082e7e") : configmap references non-existent config key: service-ca.crt Sep 15 07:22:34.785892 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:34.785746 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs podName:a951ccbf-4363-44e1-9e9b-45a3ae082e7e nodeName:}" failed. No retries permitted until 2026-09-15 07:22:42.785738495 +0000 UTC m=+243.766900640 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs") pod "router-default-c5cb56bb8-jq29f" (UID: "a951ccbf-4363-44e1-9e9b-45a3ae082e7e") : secret "router-metrics-certs-default" not found Sep 15 07:22:34.785892 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:34.785762 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 15 07:22:34.785892 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:34.785775 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b6ff5f5df-rzw29: secret "image-registry-tls" not found Sep 15 07:22:34.785892 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:34.785826 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls podName:98bd66da-f44c-42dd-b8c9-a502aa7a86c5 nodeName:}" failed. No retries permitted until 2026-09-15 07:22:42.785806675 +0000 UTC m=+243.766968839 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls") pod "image-registry-b6ff5f5df-rzw29" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5") : secret "image-registry-tls" not found Sep 15 07:22:35.087247 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.087216 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-sbzq4"] Sep 15 07:22:35.091280 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.091263 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" Sep 15 07:22:35.093504 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.093478 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 15 07:22:35.093637 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.093518 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 15 07:22:35.094334 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.094317 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 15 07:22:35.094425 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.094317 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 15 07:22:35.094425 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.094322 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-twqpd\"" Sep 15 07:22:35.097739 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.097717 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-sbzq4"] Sep 15 07:22:35.189832 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.189796 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:35.189832 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.189837 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0a7f3b80-7058-4877-a295-68b9979331d9-signing-key\") pod \"service-ca-568869fdc5-sbzq4\" (UID: \"0a7f3b80-7058-4877-a295-68b9979331d9\") " pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" Sep 15 07:22:35.190057 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:35.189959 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 07:22:35.190057 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.189982 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpt5n\" (UniqueName: \"kubernetes.io/projected/0a7f3b80-7058-4877-a295-68b9979331d9-kube-api-access-mpt5n\") pod \"service-ca-568869fdc5-sbzq4\" (UID: \"0a7f3b80-7058-4877-a295-68b9979331d9\") " pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" Sep 15 07:22:35.190057 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:35.190019 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls podName:5ef49b13-2791-4cf4-9de5-8ac40eae2eee nodeName:}" failed. No retries permitted until 2026-09-15 07:22:39.190004462 +0000 UTC m=+240.171166630 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cttzt" (UID: "5ef49b13-2791-4cf4-9de5-8ac40eae2eee") : secret "insights-runtime-extractor-tls" not found Sep 15 07:22:35.190057 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.190055 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:35.190220 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.190078 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0a7f3b80-7058-4877-a295-68b9979331d9-signing-cabundle\") pod \"service-ca-568869fdc5-sbzq4\" (UID: \"0a7f3b80-7058-4877-a295-68b9979331d9\") " pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" Sep 15 07:22:35.190220 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:35.190125 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 07:22:35.190220 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:35.190176 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm podName:5ef49b13-2791-4cf4-9de5-8ac40eae2eee nodeName:}" failed. No retries permitted until 2026-09-15 07:22:39.190163765 +0000 UTC m=+240.171325915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cttzt" (UID: "5ef49b13-2791-4cf4-9de5-8ac40eae2eee") : configmap "kube-rbac-proxy" not found Sep 15 07:22:35.290886 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.290839 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0a7f3b80-7058-4877-a295-68b9979331d9-signing-key\") pod \"service-ca-568869fdc5-sbzq4\" (UID: \"0a7f3b80-7058-4877-a295-68b9979331d9\") " pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" Sep 15 07:22:35.291039 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.290908 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpt5n\" (UniqueName: \"kubernetes.io/projected/0a7f3b80-7058-4877-a295-68b9979331d9-kube-api-access-mpt5n\") pod \"service-ca-568869fdc5-sbzq4\" (UID: \"0a7f3b80-7058-4877-a295-68b9979331d9\") " pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" Sep 15 07:22:35.291039 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.290948 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0a7f3b80-7058-4877-a295-68b9979331d9-signing-cabundle\") pod \"service-ca-568869fdc5-sbzq4\" (UID: \"0a7f3b80-7058-4877-a295-68b9979331d9\") " pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" Sep 15 07:22:35.291591 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.291568 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0a7f3b80-7058-4877-a295-68b9979331d9-signing-cabundle\") pod \"service-ca-568869fdc5-sbzq4\" (UID: \"0a7f3b80-7058-4877-a295-68b9979331d9\") " pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" Sep 15 07:22:35.293166 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.293145 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0a7f3b80-7058-4877-a295-68b9979331d9-signing-key\") pod \"service-ca-568869fdc5-sbzq4\" (UID: \"0a7f3b80-7058-4877-a295-68b9979331d9\") " pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" Sep 15 07:22:35.299197 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.299178 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpt5n\" (UniqueName: \"kubernetes.io/projected/0a7f3b80-7058-4877-a295-68b9979331d9-kube-api-access-mpt5n\") pod \"service-ca-568869fdc5-sbzq4\" (UID: \"0a7f3b80-7058-4877-a295-68b9979331d9\") " pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" Sep 15 07:22:35.370270 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.370181 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bdtkq_4169ca54-51ee-4dc7-9280-649b5143d3bf/serve-healthcheck-canary/0.log" Sep 15 07:22:35.378049 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.378014 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:35.400672 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.400648 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" Sep 15 07:22:35.678092 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:35.678031 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-sbzq4"] Sep 15 07:22:35.680645 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:22:35.680610 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a7f3b80_7058_4877_a295_68b9979331d9.slice/crio-1586f03e1f20544a9cafc6ad4096a695363fa568766043915d1fcfaff49ae070 WatchSource:0}: Error finding container 1586f03e1f20544a9cafc6ad4096a695363fa568766043915d1fcfaff49ae070: Status 404 returned error can't find the container with id 1586f03e1f20544a9cafc6ad4096a695363fa568766043915d1fcfaff49ae070 Sep 15 07:22:36.139527 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:36.139487 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:36.156776 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:36.156747 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" event={"ID":"0a7f3b80-7058-4877-a295-68b9979331d9","Type":"ContainerStarted","Data":"1586f03e1f20544a9cafc6ad4096a695363fa568766043915d1fcfaff49ae070"} Sep 15 07:22:37.401206 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:37.401119 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:37.401206 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:37.401206 2552 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:37.401576 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:37.401541 2552 scope.go:117] "RemoveContainer" containerID="88c16263b82beed42bc8848e1e0070d6d589bc75e8cd9347e8d81d9b7ffa88d8" Sep 15 07:22:37.401702 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:37.401683 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-cdwxq_openshift-console-operator(a0dd4f12-537f-42ae-99c6-2d900b956860)\"" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" podUID="a0dd4f12-537f-42ae-99c6-2d900b956860" Sep 15 07:22:38.272406 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:38.272350 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:39.221188 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:39.221137 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:39.221662 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:39.221260 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:39.221662 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:39.221264 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 07:22:39.221662 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:39.221333 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 07:22:39.221662 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:39.221384 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm podName:5ef49b13-2791-4cf4-9de5-8ac40eae2eee nodeName:}" failed. No retries permitted until 2026-09-15 07:22:47.221363723 +0000 UTC m=+248.202525882 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cttzt" (UID: "5ef49b13-2791-4cf4-9de5-8ac40eae2eee") : configmap "kube-rbac-proxy" not found Sep 15 07:22:39.221662 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:39.221403 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls podName:5ef49b13-2791-4cf4-9de5-8ac40eae2eee nodeName:}" failed. No retries permitted until 2026-09-15 07:22:47.221394279 +0000 UTC m=+248.202556424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cttzt" (UID: "5ef49b13-2791-4cf4-9de5-8ac40eae2eee") : secret "insights-runtime-extractor-tls" not found Sep 15 07:22:40.170815 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:40.170734 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" event={"ID":"0a7f3b80-7058-4877-a295-68b9979331d9","Type":"ContainerStarted","Data":"14d6d35080b3e9dc1676f80c062ea42f3a2b33bfcbaf63ab813c595047ff7e6b"} Sep 15 07:22:40.200136 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:40.200080 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-sbzq4" podStartSLOduration=1.062118561 podStartE2EDuration="5.200061797s" totalImagesPullingTime="4.137943236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:22:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:22:40.200038863 +0000 UTC m=+241.181201029" watchObservedRunningTime="2026-09-15 07:22:40.200061797 +0000 UTC m=+241.181223965" Sep 15 07:22:40.994512 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:40.994472 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:41.092474 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:41.092439 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:41.418027 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:41.417940 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:41.549497 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:41.549447 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:41.650510 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:41.650466 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:41.890033 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:41.889997 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:42.091580 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:42.091545 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:42.851334 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:42.851287 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:42.851520 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:42.851358 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls\") pod \"image-registry-b6ff5f5df-rzw29\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:42.851520 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:42.851398 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:42.851520 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:42.851446 2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 15 07:22:42.851520 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:42.851503 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 15 07:22:42.851520 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:42.851519 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b6ff5f5df-rzw29: secret "image-registry-tls" not found Sep 15 07:22:42.851671 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:42.851529 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle podName:a951ccbf-4363-44e1-9e9b-45a3ae082e7e nodeName:}" failed. No retries permitted until 2026-09-15 07:22:58.851508441 +0000 UTC m=+259.832670600 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle") pod "router-default-c5cb56bb8-jq29f" (UID: "a951ccbf-4363-44e1-9e9b-45a3ae082e7e") : configmap references non-existent config key: service-ca.crt Sep 15 07:22:42.851671 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:42.851550 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs podName:a951ccbf-4363-44e1-9e9b-45a3ae082e7e nodeName:}" failed. No retries permitted until 2026-09-15 07:22:58.851544702 +0000 UTC m=+259.832706847 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs") pod "router-default-c5cb56bb8-jq29f" (UID: "a951ccbf-4363-44e1-9e9b-45a3ae082e7e") : secret "router-metrics-certs-default" not found Sep 15 07:22:42.851671 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:42.851561 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls podName:98bd66da-f44c-42dd-b8c9-a502aa7a86c5 nodeName:}" failed. No retries permitted until 2026-09-15 07:22:58.851554851 +0000 UTC m=+259.832716996 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls") pod "image-registry-b6ff5f5df-rzw29" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5") : secret "image-registry-tls" not found Sep 15 07:22:43.082842 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:43.082807 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:43.487407 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:43.487359 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:43.687741 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:43.687662 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:44.085824 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:44.085785 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:45.706340 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:45.706309 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:45.884178 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:45.884144 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:46.686999 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:46.686968 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:46.882691 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:46.882654 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:47.095950 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:47.095918 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:47.288396 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:47.288354 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:47.288563 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:47.288544 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 07:22:47.288643 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:47.288632 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls podName:5ef49b13-2791-4cf4-9de5-8ac40eae2eee nodeName:}" failed. No retries permitted until 2026-09-15 07:23:03.28860973 +0000 UTC m=+264.269771880 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cttzt" (UID: "5ef49b13-2791-4cf4-9de5-8ac40eae2eee") : secret "insights-runtime-extractor-tls" not found Sep 15 07:22:47.288704 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:47.288542 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:47.289141 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:47.289119 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:22:47.295687 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:47.295656 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:47.400955 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:47.400850 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:47.508188 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:47.508154 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:48.086316 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:48.086281 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:48.681353 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:48.681312 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:48.889600 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:48.889568 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:49.084616 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:49.084584 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:49.409464 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:49.409376 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:49.500538 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:49.500483 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:49.561229 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:49.561196 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:50.094745 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:50.094712 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:50.468590 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:50.468509 2552 scope.go:117] "RemoveContainer" containerID="88c16263b82beed42bc8848e1e0070d6d589bc75e8cd9347e8d81d9b7ffa88d8" Sep 15 07:22:50.485321 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:50.485290 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:50.885310 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:50.885280 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:51.085663 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.085627 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:51.209553 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.209486 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:22:51.209902 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.209861 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/1.log" Sep 15 07:22:51.210013 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.209922 2552 generic.go:356] "Generic (PLEG): container finished" podID="a0dd4f12-537f-42ae-99c6-2d900b956860" containerID="98871f96638575065b4ed10807cf5e79f7a49d7db7faf372b107e9afc8bd348b" exitCode=255 Sep 15 07:22:51.210013 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.209997 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" event={"ID":"a0dd4f12-537f-42ae-99c6-2d900b956860","Type":"ContainerDied","Data":"98871f96638575065b4ed10807cf5e79f7a49d7db7faf372b107e9afc8bd348b"} Sep 15 07:22:51.210090 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.210048 2552 scope.go:117] "RemoveContainer" containerID="88c16263b82beed42bc8848e1e0070d6d589bc75e8cd9347e8d81d9b7ffa88d8" Sep 15 07:22:51.210386 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.210371 2552 scope.go:117] "RemoveContainer" containerID="98871f96638575065b4ed10807cf5e79f7a49d7db7faf372b107e9afc8bd348b" Sep 15 07:22:51.210554 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:51.210537 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-cdwxq_openshift-console-operator(a0dd4f12-537f-42ae-99c6-2d900b956860)\"" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" podUID="a0dd4f12-537f-42ae-99c6-2d900b956860" Sep 15 07:22:51.216906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.216884 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:22:51.219038 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.218923 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13cc492-490d-478f-b799-6127572cc075-metrics-certs\") pod \"network-metrics-daemon-n9rxl\" (UID: \"f13cc492-490d-478f-b799-6127572cc075\") " pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:22:51.471195 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.471115 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ftj64\"" Sep 15 07:22:51.478925 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.478904 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9rxl" Sep 15 07:22:51.482408 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.482288 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:51.682628 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.682596 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:51.747754 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:51.747726 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n9rxl"] Sep 15 07:22:51.750399 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:22:51.750375 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13cc492_490d_478f_b799_6127572cc075.slice/crio-5e2f53c28e81cd32a38983dcb0a48ebc8f79c86be51be183b32405e3210236c4 WatchSource:0}: Error finding container 5e2f53c28e81cd32a38983dcb0a48ebc8f79c86be51be183b32405e3210236c4: Status 404 returned error can't find the container with id 5e2f53c28e81cd32a38983dcb0a48ebc8f79c86be51be183b32405e3210236c4 Sep 15 07:22:52.214312 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:52.214239 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:22:52.215418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:52.215389 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n9rxl" event={"ID":"f13cc492-490d-478f-b799-6127572cc075","Type":"ContainerStarted","Data":"5e2f53c28e81cd32a38983dcb0a48ebc8f79c86be51be183b32405e3210236c4"} Sep 15 07:22:53.025605 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:53.025574 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:53.126042 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:53.126012 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:53.220231 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:53.220149 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n9rxl" event={"ID":"f13cc492-490d-478f-b799-6127572cc075","Type":"ContainerStarted","Data":"b72e6a958c13dbb57ca3740c7c3afc2ebc547d18c7b5b8e48fb2f6a2fa098a22"} Sep 15 07:22:53.220231 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:53.220229 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n9rxl" event={"ID":"f13cc492-490d-478f-b799-6127572cc075","Type":"ContainerStarted","Data":"b39c1a83e2c4148926b8c0e8fb45002c61ca81cd9fc39ba3c86827eeffddcbd9"} Sep 15 07:22:53.236523 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:53.236468 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-n9rxl" podStartSLOduration=253.07584843 podStartE2EDuration="4m14.236452712s" totalImagesPullingTime="1.160604282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:22:53.235286064 +0000 UTC m=+254.216448232" watchObservedRunningTime="2026-09-15 07:22:53.236452712 +0000 UTC m=+254.217614880" Sep 15 07:22:54.557985 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:54.557951 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b6ff5f5df-rzw29"] Sep 15 07:22:54.558344 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:54.558171 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" podUID="98bd66da-f44c-42dd-b8c9-a502aa7a86c5" Sep 15 07:22:54.660658 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:54.660619 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:54.763324 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:54.763296 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:22:55.226353 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.226322 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:55.231533 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.231511 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:55.352517 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.352482 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-certificates\") pod \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " Sep 15 07:22:55.352680 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.352530 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-image-registry-private-configuration\") pod \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " Sep 15 07:22:55.352680 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.352576 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-bound-sa-token\") pod \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " Sep 15 07:22:55.352680 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.352606 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-ca-trust-extracted\") pod \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " Sep 15 07:22:55.352680 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.352624 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-installation-pull-secrets\") pod \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " Sep 15 07:22:55.352680 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.352668 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7ss6\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-kube-api-access-r7ss6\") pod \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " Sep 15 07:22:55.352951 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.352694 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-trusted-ca\") pod \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\" (UID: \"98bd66da-f44c-42dd-b8c9-a502aa7a86c5\") " Sep 15 07:22:55.352951 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.352853 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "98bd66da-f44c-42dd-b8c9-a502aa7a86c5" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:22:55.353065 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.353006 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "98bd66da-f44c-42dd-b8c9-a502aa7a86c5" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:22:55.353120 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.353083 2552 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-certificates\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:22:55.353281 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.353249 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "98bd66da-f44c-42dd-b8c9-a502aa7a86c5" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:22:55.354776 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.354756 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "98bd66da-f44c-42dd-b8c9-a502aa7a86c5" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:22:55.354931 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.354911 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "98bd66da-f44c-42dd-b8c9-a502aa7a86c5" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:22:55.355162 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.355137 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-kube-api-access-r7ss6" (OuterVolumeSpecName: "kube-api-access-r7ss6") pod "98bd66da-f44c-42dd-b8c9-a502aa7a86c5" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5"). InnerVolumeSpecName "kube-api-access-r7ss6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:22:55.355270 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.355236 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "98bd66da-f44c-42dd-b8c9-a502aa7a86c5" (UID: "98bd66da-f44c-42dd-b8c9-a502aa7a86c5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:22:55.453974 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.453932 2552 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-image-registry-private-configuration\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:22:55.453974 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.453961 2552 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-bound-sa-token\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:22:55.453974 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.453972 2552 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-ca-trust-extracted\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:22:55.453974 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.453982 2552 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-installation-pull-secrets\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:22:55.454212 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.453991 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7ss6\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-kube-api-access-r7ss6\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:22:55.454212 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:55.454000 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-trusted-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:22:56.230143 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:56.230114 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b6ff5f5df-rzw29" Sep 15 07:22:56.261488 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:56.261458 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b6ff5f5df-rzw29"] Sep 15 07:22:56.264628 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:56.264597 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b6ff5f5df-rzw29"] Sep 15 07:22:56.360555 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:56.360525 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98bd66da-f44c-42dd-b8c9-a502aa7a86c5-registry-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:22:57.401099 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:57.401061 2552 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:57.401489 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:57.401118 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:22:57.401489 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:57.401405 2552 scope.go:117] "RemoveContainer" containerID="98871f96638575065b4ed10807cf5e79f7a49d7db7faf372b107e9afc8bd348b" Sep 15 07:22:57.401584 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:22:57.401566 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-cdwxq_openshift-console-operator(a0dd4f12-537f-42ae-99c6-2d900b956860)\"" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" podUID="a0dd4f12-537f-42ae-99c6-2d900b956860" Sep 15 07:22:57.472655 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:57.472624 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98bd66da-f44c-42dd-b8c9-a502aa7a86c5" path="/var/lib/kubelet/pods/98bd66da-f44c-42dd-b8c9-a502aa7a86c5/volumes" Sep 15 07:22:58.877529 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:58.877482 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:58.877955 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:58.877655 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:58.878230 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:58.878214 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-service-ca-bundle\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:58.879813 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:58.879793 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a951ccbf-4363-44e1-9e9b-45a3ae082e7e-metrics-certs\") pod \"router-default-c5cb56bb8-jq29f\" (UID: \"a951ccbf-4363-44e1-9e9b-45a3ae082e7e\") " pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:59.103162 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:59.103127 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8qj4j\"" Sep 15 07:22:59.111206 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:59.111183 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:22:59.384334 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:22:59.384303 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c5cb56bb8-jq29f"] Sep 15 07:22:59.387532 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:22:59.387503 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda951ccbf_4363_44e1_9e9b_45a3ae082e7e.slice/crio-781aa388ce2e61af5674ee6cc66a62e8058179f4a3176e357ef605ce42558454 WatchSource:0}: Error finding container 781aa388ce2e61af5674ee6cc66a62e8058179f4a3176e357ef605ce42558454: Status 404 returned error can't find the container with id 781aa388ce2e61af5674ee6cc66a62e8058179f4a3176e357ef605ce42558454 Sep 15 07:23:00.243883 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:00.243848 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c5cb56bb8-jq29f" event={"ID":"a951ccbf-4363-44e1-9e9b-45a3ae082e7e","Type":"ContainerStarted","Data":"70e4e75194d81559b54820d491e8641ba897b6e0465db87bdd27e1bfc3953f1e"} Sep 15 07:23:00.244334 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:00.243907 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c5cb56bb8-jq29f" event={"ID":"a951ccbf-4363-44e1-9e9b-45a3ae082e7e","Type":"ContainerStarted","Data":"781aa388ce2e61af5674ee6cc66a62e8058179f4a3176e357ef605ce42558454"} Sep 15 07:23:00.265314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:00.265266 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-c5cb56bb8-jq29f" podStartSLOduration=34.265249813 podStartE2EDuration="34.265249813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:22:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:23:00.263449461 +0000 UTC m=+261.244611627" watchObservedRunningTime="2026-09-15 07:23:00.265249813 +0000 UTC m=+261.246411979" Sep 15 07:23:01.112439 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:01.112390 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:23:01.114906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:01.114881 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:23:01.247356 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:01.247328 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:23:01.248550 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:01.248526 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-c5cb56bb8-jq29f" Sep 15 07:23:01.929792 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:01.929706 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:02.029145 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.029107 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:02.132282 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.132251 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:02.173274 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.173239 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd"] Sep 15 07:23:02.175334 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.175318 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd" Sep 15 07:23:02.177428 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.177405 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 15 07:23:02.177516 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.177438 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-pjwpd\"" Sep 15 07:23:02.183988 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.183966 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd"] Sep 15 07:23:02.197575 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.197542 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:02.305159 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.305129 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2e6011ff-a0de-44c7-8899-d0b57b5a2e88-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-msdwd\" (UID: \"2e6011ff-a0de-44c7-8899-d0b57b5a2e88\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd" Sep 15 07:23:02.349207 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.349172 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:02.405730 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.405700 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2e6011ff-a0de-44c7-8899-d0b57b5a2e88-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-msdwd\" (UID: \"2e6011ff-a0de-44c7-8899-d0b57b5a2e88\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd" Sep 15 07:23:02.408181 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.408159 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2e6011ff-a0de-44c7-8899-d0b57b5a2e88-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-msdwd\" (UID: \"2e6011ff-a0de-44c7-8899-d0b57b5a2e88\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd" Sep 15 07:23:02.484500 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.484469 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd" Sep 15 07:23:02.753909 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:02.753809 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd"] Sep 15 07:23:02.757857 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:23:02.757826 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e6011ff_a0de_44c7_8899_d0b57b5a2e88.slice/crio-5d3787c3eca114af25f47d3925197fa4fb18a10e5d8a8d109db8cbd86b3bdcd4 WatchSource:0}: Error finding container 5d3787c3eca114af25f47d3925197fa4fb18a10e5d8a8d109db8cbd86b3bdcd4: Status 404 returned error can't find the container with id 5d3787c3eca114af25f47d3925197fa4fb18a10e5d8a8d109db8cbd86b3bdcd4 Sep 15 07:23:03.254963 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:03.254927 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd" event={"ID":"2e6011ff-a0de-44c7-8899-d0b57b5a2e88","Type":"ContainerStarted","Data":"5d3787c3eca114af25f47d3925197fa4fb18a10e5d8a8d109db8cbd86b3bdcd4"} Sep 15 07:23:03.312365 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:03.312333 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:23:03.314656 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:03.314635 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ef49b13-2791-4cf4-9de5-8ac40eae2eee-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cttzt\" (UID: \"5ef49b13-2791-4cf4-9de5-8ac40eae2eee\") " pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:23:03.536678 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:03.536600 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mn7st\"" Sep 15 07:23:03.543479 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:03.543462 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cttzt" Sep 15 07:23:03.808967 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:03.808755 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cttzt"] Sep 15 07:23:03.811467 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:23:03.811438 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef49b13_2791_4cf4_9de5_8ac40eae2eee.slice/crio-bf8265104cbd2c7c09cbb1b029e8a1272a71b63d99aaf2ab940e45d040c9b11a WatchSource:0}: Error finding container bf8265104cbd2c7c09cbb1b029e8a1272a71b63d99aaf2ab940e45d040c9b11a: Status 404 returned error can't find the container with id bf8265104cbd2c7c09cbb1b029e8a1272a71b63d99aaf2ab940e45d040c9b11a Sep 15 07:23:04.258926 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:04.258895 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cttzt" event={"ID":"5ef49b13-2791-4cf4-9de5-8ac40eae2eee","Type":"ContainerStarted","Data":"c188e68ab49a82dbe694c96a514d500dca49aad7ab09e8acc4cc3c71b0e0c35b"} Sep 15 07:23:04.259094 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:04.258956 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cttzt" event={"ID":"5ef49b13-2791-4cf4-9de5-8ac40eae2eee","Type":"ContainerStarted","Data":"bf8265104cbd2c7c09cbb1b029e8a1272a71b63d99aaf2ab940e45d040c9b11a"} Sep 15 07:23:05.264882 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:05.264835 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cttzt" event={"ID":"5ef49b13-2791-4cf4-9de5-8ac40eae2eee","Type":"ContainerStarted","Data":"969ddd2d88590a7954441badb7d395967c4c6d032c2e630835b572ed6ad0fe0f"} Sep 15 07:23:07.274570 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:07.274528 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cttzt" event={"ID":"5ef49b13-2791-4cf4-9de5-8ac40eae2eee","Type":"ContainerStarted","Data":"dd3ad61bc6c0cb04048be6df063b3ab56987f35ee839a817759d677e1143dcfc"} Sep 15 07:23:07.292462 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:07.292405 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cttzt" podStartSLOduration=33.494768462 podStartE2EDuration="36.292377539s" totalImagesPullingTime="2.797609077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:22:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:23:07.290857714 +0000 UTC m=+268.272019882" watchObservedRunningTime="2026-09-15 07:23:07.292377539 +0000 UTC m=+268.273539707" Sep 15 07:23:08.279590 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:08.279554 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd" event={"ID":"2e6011ff-a0de-44c7-8899-d0b57b5a2e88","Type":"ContainerStarted","Data":"50cf3819d7261d7764f4adbcf69098d8a6aa856e61b94dda24a71aa519772dcf"} Sep 15 07:23:08.280075 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:08.279778 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd" Sep 15 07:23:08.284435 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:08.284408 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd" Sep 15 07:23:08.294089 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:08.294037 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-msdwd" podStartSLOduration=1.357144092 podStartE2EDuration="6.294023356s" totalImagesPullingTime="4.936879264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:23:08.293793043 +0000 UTC m=+269.274955210" watchObservedRunningTime="2026-09-15 07:23:08.294023356 +0000 UTC m=+269.275185522" Sep 15 07:23:09.206963 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.206781 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:09.237673 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.237646 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-l42tm"] Sep 15 07:23:09.239963 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.239948 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.242337 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.242314 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lqw46\"" Sep 15 07:23:09.243432 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.243412 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 15 07:23:09.243531 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.243441 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 15 07:23:09.243531 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.243455 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 15 07:23:09.243651 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.243533 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 15 07:23:09.243744 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.243729 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 15 07:23:09.250047 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.250028 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-l42tm"] Sep 15 07:23:09.302768 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.302740 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:09.357977 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.357952 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lqxn\" (UniqueName: \"kubernetes.io/projected/3ad379ee-05df-4ab2-b2b3-302669226c77-kube-api-access-6lqxn\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.358064 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.357987 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ad379ee-05df-4ab2-b2b3-302669226c77-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.358064 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.358027 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ad379ee-05df-4ab2-b2b3-302669226c77-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.358064 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.358058 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ad379ee-05df-4ab2-b2b3-302669226c77-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.458380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.458308 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ad379ee-05df-4ab2-b2b3-302669226c77-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.458380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.458347 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lqxn\" (UniqueName: \"kubernetes.io/projected/3ad379ee-05df-4ab2-b2b3-302669226c77-kube-api-access-6lqxn\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.458380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.458378 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ad379ee-05df-4ab2-b2b3-302669226c77-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.458641 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.458426 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ad379ee-05df-4ab2-b2b3-302669226c77-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.458641 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:23:09.458457 2552 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 15 07:23:09.458641 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:23:09.458524 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ad379ee-05df-4ab2-b2b3-302669226c77-prometheus-operator-tls podName:3ad379ee-05df-4ab2-b2b3-302669226c77 nodeName:}" failed. No retries permitted until 2026-09-15 07:23:09.958503023 +0000 UTC m=+270.939665187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3ad379ee-05df-4ab2-b2b3-302669226c77-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-l42tm" (UID: "3ad379ee-05df-4ab2-b2b3-302669226c77") : secret "prometheus-operator-tls" not found Sep 15 07:23:09.459255 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.459221 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ad379ee-05df-4ab2-b2b3-302669226c77-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.460776 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.460754 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ad379ee-05df-4ab2-b2b3-302669226c77-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.467447 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.467424 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lqxn\" (UniqueName: \"kubernetes.io/projected/3ad379ee-05df-4ab2-b2b3-302669226c77-kube-api-access-6lqxn\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.470478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.470456 2552 scope.go:117] "RemoveContainer" containerID="98871f96638575065b4ed10807cf5e79f7a49d7db7faf372b107e9afc8bd348b" Sep 15 07:23:09.470650 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:23:09.470628 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-cdwxq_openshift-console-operator(a0dd4f12-537f-42ae-99c6-2d900b956860)\"" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" podUID="a0dd4f12-537f-42ae-99c6-2d900b956860" Sep 15 07:23:09.902527 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.902493 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:09.962781 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.962750 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ad379ee-05df-4ab2-b2b3-302669226c77-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:09.965268 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:09.965243 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3ad379ee-05df-4ab2-b2b3-302669226c77-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-l42tm\" (UID: \"3ad379ee-05df-4ab2-b2b3-302669226c77\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:10.148395 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:10.148361 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" Sep 15 07:23:10.416485 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:10.416411 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-l42tm"] Sep 15 07:23:10.419747 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:23:10.419722 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ad379ee_05df_4ab2_b2b3_302669226c77.slice/crio-2e8c808632298e9c6acaa2aa32ab799b5563502746a3c0289c2d352bd53417db WatchSource:0}: Error finding container 2e8c808632298e9c6acaa2aa32ab799b5563502746a3c0289c2d352bd53417db: Status 404 returned error can't find the container with id 2e8c808632298e9c6acaa2aa32ab799b5563502746a3c0289c2d352bd53417db Sep 15 07:23:11.290421 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:11.290384 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" event={"ID":"3ad379ee-05df-4ab2-b2b3-302669226c77","Type":"ContainerStarted","Data":"2e8c808632298e9c6acaa2aa32ab799b5563502746a3c0289c2d352bd53417db"} Sep 15 07:23:11.951665 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:11.951581 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:12.295901 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:12.295859 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" event={"ID":"3ad379ee-05df-4ab2-b2b3-302669226c77","Type":"ContainerStarted","Data":"99b308309c0aa7b157fb6c4e28180e21934e476214c9c8c66b39bf977ea2ee21"} Sep 15 07:23:12.296063 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:12.295922 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" event={"ID":"3ad379ee-05df-4ab2-b2b3-302669226c77","Type":"ContainerStarted","Data":"d68a649c47cf71c958cbe95000e5b4f27756e937f4efe499e22ce8196d0063b4"} Sep 15 07:23:12.312513 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:12.312471 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l42tm" podStartSLOduration=2.032813795 podStartE2EDuration="3.312457056s" totalImagesPullingTime="1.279643261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:23:12.31111388 +0000 UTC m=+273.292276072" watchObservedRunningTime="2026-09-15 07:23:12.312457056 +0000 UTC m=+273.293619223" Sep 15 07:23:14.350497 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.350454 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:14.443655 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.443622 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:14.544174 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.544137 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:14.590920 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.590883 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lxcpq"] Sep 15 07:23:14.594436 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.594416 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.596761 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.596735 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9ch4s\"" Sep 15 07:23:14.596917 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.596830 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 15 07:23:14.596917 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.596844 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 15 07:23:14.596917 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.596856 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 15 07:23:14.682311 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.682280 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:14.694455 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.694427 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.694558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.694472 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-tls\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.694558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.694500 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.694642 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.694556 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqxt2\" (UniqueName: \"kubernetes.io/projected/ea4664a6-2ac7-4a77-bd97-91e53c772091-kube-api-access-dqxt2\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.694642 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.694633 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea4664a6-2ac7-4a77-bd97-91e53c772091-sys\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.694715 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.694660 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea4664a6-2ac7-4a77-bd97-91e53c772091-metrics-client-ca\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.694715 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.694680 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ea4664a6-2ac7-4a77-bd97-91e53c772091-root\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.694715 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.694699 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-wtmp\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.694810 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.694771 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-textfile\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.746940 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.746908 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:14.795050 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795023 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-wtmp\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795170 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795055 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-textfile\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795170 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795083 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795170 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795108 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-tls\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795170 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795126 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795209 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-wtmp\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795259 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqxt2\" (UniqueName: \"kubernetes.io/projected/ea4664a6-2ac7-4a77-bd97-91e53c772091-kube-api-access-dqxt2\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795305 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea4664a6-2ac7-4a77-bd97-91e53c772091-sys\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795348 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea4664a6-2ac7-4a77-bd97-91e53c772091-metrics-client-ca\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795355 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea4664a6-2ac7-4a77-bd97-91e53c772091-sys\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795600 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795381 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ea4664a6-2ac7-4a77-bd97-91e53c772091-root\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795600 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795453 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ea4664a6-2ac7-4a77-bd97-91e53c772091-root\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795698 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795679 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-textfile\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.795824 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.795807 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.796626 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.796611 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea4664a6-2ac7-4a77-bd97-91e53c772091-metrics-client-ca\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.797398 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.797378 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.797490 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.797473 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ea4664a6-2ac7-4a77-bd97-91e53c772091-node-exporter-tls\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.802624 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.802604 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqxt2\" (UniqueName: \"kubernetes.io/projected/ea4664a6-2ac7-4a77-bd97-91e53c772091-kube-api-access-dqxt2\") pod \"node-exporter-lxcpq\" (UID: \"ea4664a6-2ac7-4a77-bd97-91e53c772091\") " pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.803488 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.803464 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:14.903800 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.903719 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lxcpq" Sep 15 07:23:14.914898 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:23:14.914856 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4664a6_2ac7_4a77_bd97_91e53c772091.slice/crio-098be09ee688b2df6a8272d75f0f90ef7edc42dc8d600f9cab53b10e4614d325 WatchSource:0}: Error finding container 098be09ee688b2df6a8272d75f0f90ef7edc42dc8d600f9cab53b10e4614d325: Status 404 returned error can't find the container with id 098be09ee688b2df6a8272d75f0f90ef7edc42dc8d600f9cab53b10e4614d325 Sep 15 07:23:14.947816 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:14.947786 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:15.044886 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.044839 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:15.308506 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.308476 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxcpq" event={"ID":"ea4664a6-2ac7-4a77-bd97-91e53c772091","Type":"ContainerStarted","Data":"098be09ee688b2df6a8272d75f0f90ef7edc42dc8d600f9cab53b10e4614d325"} Sep 15 07:23:15.592596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.592499 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:15.670730 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.670700 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:15.677037 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.677004 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:23:15.680797 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.680751 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.685274 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.685249 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 15 07:23:15.685627 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.685536 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 15 07:23:15.685731 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.685665 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 15 07:23:15.685785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.685759 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 15 07:23:15.685923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.685905 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 15 07:23:15.686230 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.686118 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2jf6v\"" Sep 15 07:23:15.686230 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.686185 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 15 07:23:15.686350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.686319 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 15 07:23:15.686407 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.686351 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 15 07:23:15.686407 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.686377 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 15 07:23:15.698236 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.698155 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:23:15.700596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.700563 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.701202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.700605 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5nfl\" (UniqueName: \"kubernetes.io/projected/c0491b87-e447-4e42-aa9d-281631cd6aab-kube-api-access-n5nfl\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.701202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.700634 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.701202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.700663 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.701202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.700681 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-web-config\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.701202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.700698 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-config-volume\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.701202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.700849 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.701202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.700895 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0491b87-e447-4e42-aa9d-281631cd6aab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.701202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.700926 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.701202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.700964 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.701202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.700986 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.701202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.701014 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0491b87-e447-4e42-aa9d-281631cd6aab-config-out\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.701202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.701037 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.793324 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.793281 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:15.801439 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.801408 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.801576 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.801465 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.801576 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.801493 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.801576 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.801523 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0491b87-e447-4e42-aa9d-281631cd6aab-config-out\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.801576 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.801546 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.801768 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.801689 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.801768 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.801737 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5nfl\" (UniqueName: \"kubernetes.io/projected/c0491b87-e447-4e42-aa9d-281631cd6aab-kube-api-access-n5nfl\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.801891 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.801767 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.801891 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.801799 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.801891 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.801828 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-web-config\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.801891 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.801859 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-config-volume\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.802107 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:23:15.802057 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-trusted-ca-bundle podName:c0491b87-e447-4e42-aa9d-281631cd6aab nodeName:}" failed. No retries permitted until 2026-09-15 07:23:16.302032879 +0000 UTC m=+277.283195032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab") : configmap references non-existent config key: ca-bundle.crt Sep 15 07:23:15.802278 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.802245 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.802332 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.802299 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0491b87-e447-4e42-aa9d-281631cd6aab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.802390 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.802374 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.802662 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.802637 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.805158 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.804656 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.805158 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.804683 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0491b87-e447-4e42-aa9d-281631cd6aab-config-out\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.805158 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.805105 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-config-volume\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.805158 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.805130 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.805368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.805307 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-web-config\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.805368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.805314 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.805479 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.805456 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.805727 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.805707 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.806155 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.806137 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0491b87-e447-4e42-aa9d-281631cd6aab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:15.810506 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:15.810476 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5nfl\" (UniqueName: \"kubernetes.io/projected/c0491b87-e447-4e42-aa9d-281631cd6aab-kube-api-access-n5nfl\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:16.306811 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:16.306776 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:16.307829 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:16.307807 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:16.312917 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:16.312887 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxcpq" event={"ID":"ea4664a6-2ac7-4a77-bd97-91e53c772091","Type":"ContainerStarted","Data":"91b5b21111ee1f0c943d09ed63cab74ff9f1127418473142bf224567f15fbb80"} Sep 15 07:23:16.593633 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:16.593604 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:23:16.869984 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:16.869961 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:23:16.872342 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:23:16.872310 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0491b87_e447_4e42_aa9d_281631cd6aab.slice/crio-2a32beedb37aa0a229f44e4576707ca2ba2c96e5df5945c96104af45158124d4 WatchSource:0}: Error finding container 2a32beedb37aa0a229f44e4576707ca2ba2c96e5df5945c96104af45158124d4: Status 404 returned error can't find the container with id 2a32beedb37aa0a229f44e4576707ca2ba2c96e5df5945c96104af45158124d4 Sep 15 07:23:17.321926 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:17.321888 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerStarted","Data":"2a32beedb37aa0a229f44e4576707ca2ba2c96e5df5945c96104af45158124d4"} Sep 15 07:23:17.323405 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:17.323379 2552 generic.go:356] "Generic (PLEG): container finished" podID="ea4664a6-2ac7-4a77-bd97-91e53c772091" containerID="91b5b21111ee1f0c943d09ed63cab74ff9f1127418473142bf224567f15fbb80" exitCode=0 Sep 15 07:23:17.323526 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:17.323424 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxcpq" event={"ID":"ea4664a6-2ac7-4a77-bd97-91e53c772091","Type":"ContainerDied","Data":"91b5b21111ee1f0c943d09ed63cab74ff9f1127418473142bf224567f15fbb80"} Sep 15 07:23:17.344383 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:17.344354 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:17.442284 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:17.442257 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:17.543058 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:17.543027 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:17.681088 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:17.681003 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:17.995264 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:17.995218 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:18.120740 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:18.120469 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:18.329594 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:18.329516 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxcpq" event={"ID":"ea4664a6-2ac7-4a77-bd97-91e53c772091","Type":"ContainerStarted","Data":"f55fd0e805c04acc568f7b3d31ce890ada99f63df09aeb2d1e40a0406b6a5c5d"} Sep 15 07:23:18.329594 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:18.329577 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxcpq" event={"ID":"ea4664a6-2ac7-4a77-bd97-91e53c772091","Type":"ContainerStarted","Data":"c7cf948fa4003dfb2156f6e2d2a73b1a5ba2bc8cbe5ce29de0a28e3bd3788685"} Sep 15 07:23:18.350372 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:18.350325 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lxcpq" podStartSLOduration=2.99431211 podStartE2EDuration="4.350308513s" totalImagesPullingTime="1.294541236s" totalInitContainerRuntime="61.455167ms" isStatefulPod=true podCreationTimestamp="2026-09-15 07:23:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:23:18.34881447 +0000 UTC m=+279.329976638" watchObservedRunningTime="2026-09-15 07:23:18.350308513 +0000 UTC m=+279.331470679" Sep 15 07:23:18.922793 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:23:18.922706 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bsrr6" podUID="ee9505e1-41b8-4f77-b211-c41076a8eac9" Sep 15 07:23:19.336007 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:19.335977 2552 generic.go:356] "Generic (PLEG): container finished" podID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerID="a47c3b61a885ac5095539fc249510388181dcebe470c30d55f894b0dd6911cc9" exitCode=0 Sep 15 07:23:19.336180 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:19.336064 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerDied","Data":"a47c3b61a885ac5095539fc249510388181dcebe470c30d55f894b0dd6911cc9"} Sep 15 07:23:19.336344 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:19.336307 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bsrr6" Sep 15 07:23:19.390844 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:19.390815 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:19.708731 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:19.708637 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:20.766584 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:20.766549 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:22.754290 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:22.754253 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:23:22.756562 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:22.756532 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee9505e1-41b8-4f77-b211-c41076a8eac9-metrics-tls\") pod \"dns-default-bsrr6\" (UID: \"ee9505e1-41b8-4f77-b211-c41076a8eac9\") " pod="openshift-dns/dns-default-bsrr6" Sep 15 07:23:22.939927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:22.939894 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tfj9f\"" Sep 15 07:23:22.947304 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:22.947279 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:22.948019 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:22.947988 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bsrr6" Sep 15 07:23:23.005421 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:23.005314 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:23.216967 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:23.216929 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bsrr6"] Sep 15 07:23:23.219618 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:23:23.219590 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee9505e1_41b8_4f77_b211_c41076a8eac9.slice/crio-7d5bd308547cc32ab8ea18c307cfd638b1c0c74b6756c39e67b112c7592acdd4 WatchSource:0}: Error finding container 7d5bd308547cc32ab8ea18c307cfd638b1c0c74b6756c39e67b112c7592acdd4: Status 404 returned error can't find the container with id 7d5bd308547cc32ab8ea18c307cfd638b1c0c74b6756c39e67b112c7592acdd4 Sep 15 07:23:23.349742 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:23.349670 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bsrr6" event={"ID":"ee9505e1-41b8-4f77-b211-c41076a8eac9","Type":"ContainerStarted","Data":"7d5bd308547cc32ab8ea18c307cfd638b1c0c74b6756c39e67b112c7592acdd4"} Sep 15 07:23:23.469329 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:23.469298 2552 scope.go:117] "RemoveContainer" containerID="98871f96638575065b4ed10807cf5e79f7a49d7db7faf372b107e9afc8bd348b" Sep 15 07:23:24.045541 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.045505 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-q67fs"] Sep 15 07:23:24.049224 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.049198 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-q67fs" Sep 15 07:23:24.051569 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.051418 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 15 07:23:24.051569 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.051464 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 15 07:23:24.051844 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.051741 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-76d5r\"" Sep 15 07:23:24.059093 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.059020 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-q67fs"] Sep 15 07:23:24.063276 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.063132 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn9gp\" (UniqueName: \"kubernetes.io/projected/e617ad16-f2c5-41a9-b78b-e2936a79f16b-kube-api-access-rn9gp\") pod \"downloads-76f99584f4-q67fs\" (UID: \"e617ad16-f2c5-41a9-b78b-e2936a79f16b\") " pod="openshift-console/downloads-76f99584f4-q67fs" Sep 15 07:23:24.063276 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.063165 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:24.120454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.120374 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:24.164172 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.164140 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn9gp\" (UniqueName: \"kubernetes.io/projected/e617ad16-f2c5-41a9-b78b-e2936a79f16b-kube-api-access-rn9gp\") pod \"downloads-76f99584f4-q67fs\" (UID: \"e617ad16-f2c5-41a9-b78b-e2936a79f16b\") " pod="openshift-console/downloads-76f99584f4-q67fs" Sep 15 07:23:24.172323 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.172294 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn9gp\" (UniqueName: \"kubernetes.io/projected/e617ad16-f2c5-41a9-b78b-e2936a79f16b-kube-api-access-rn9gp\") pod \"downloads-76f99584f4-q67fs\" (UID: \"e617ad16-f2c5-41a9-b78b-e2936a79f16b\") " pod="openshift-console/downloads-76f99584f4-q67fs" Sep 15 07:23:24.180099 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.180072 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:24.355790 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.355769 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:23:24.355923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.355899 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" event={"ID":"a0dd4f12-537f-42ae-99c6-2d900b956860","Type":"ContainerStarted","Data":"12e1ae46b548bcb440bb56b30fcf4c9396d093471017e8335e3cfc81de7d98be"} Sep 15 07:23:24.356339 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.356177 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:23:24.359788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.359761 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:24.361599 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.361580 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-q67fs" Sep 15 07:23:24.362302 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.362257 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" Sep 15 07:23:24.373666 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.373618 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-cdwxq" podStartSLOduration=54.877588582 podStartE2EDuration="57.373601966s" totalImagesPullingTime="2.496013384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:23:24.372811985 +0000 UTC m=+285.353974153" watchObservedRunningTime="2026-09-15 07:23:24.373601966 +0000 UTC m=+285.354764132" Sep 15 07:23:24.529621 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.529413 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:24.590264 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.590182 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:24.645691 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.645655 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-q67fs"] Sep 15 07:23:24.886238 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:23:24.886169 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode617ad16_f2c5_41a9_b78b_e2936a79f16b.slice/crio-73dbfafb860885e0d1acdc9e1cee19217a877a6e3b8009c622e713ea593d116c WatchSource:0}: Error finding container 73dbfafb860885e0d1acdc9e1cee19217a877a6e3b8009c622e713ea593d116c: Status 404 returned error can't find the container with id 73dbfafb860885e0d1acdc9e1cee19217a877a6e3b8009c622e713ea593d116c Sep 15 07:23:24.948801 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:24.948772 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:25.361183 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:25.361146 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bsrr6" event={"ID":"ee9505e1-41b8-4f77-b211-c41076a8eac9","Type":"ContainerStarted","Data":"33a92e5c34544a603d72c7754fbb0c8bccc885d622ecda4bc26bd21ba3df3a74"} Sep 15 07:23:25.361595 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:25.361243 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bsrr6" Sep 15 07:23:25.361595 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:25.361269 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bsrr6" event={"ID":"ee9505e1-41b8-4f77-b211-c41076a8eac9","Type":"ContainerStarted","Data":"84d6c85a2e0fc7df17a48c3baf40e55009be256e9deba3ffa69745a7dd84a4d8"} Sep 15 07:23:25.362399 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:25.362372 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-q67fs" event={"ID":"e617ad16-f2c5-41a9-b78b-e2936a79f16b","Type":"ContainerStarted","Data":"73dbfafb860885e0d1acdc9e1cee19217a877a6e3b8009c622e713ea593d116c"} Sep 15 07:23:25.365446 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:25.365417 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerStarted","Data":"c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d"} Sep 15 07:23:25.365555 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:25.365499 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerStarted","Data":"1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b"} Sep 15 07:23:25.365555 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:25.365542 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerStarted","Data":"140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099"} Sep 15 07:23:25.365662 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:25.365588 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerStarted","Data":"2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c"} Sep 15 07:23:25.365662 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:25.365630 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerStarted","Data":"66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763"} Sep 15 07:23:25.380430 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:25.380388 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bsrr6" podStartSLOduration=251.666740867 podStartE2EDuration="4m13.380374037s" totalImagesPullingTime="1.71363317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:23:25.378943857 +0000 UTC m=+286.360106025" watchObservedRunningTime="2026-09-15 07:23:25.380374037 +0000 UTC m=+286.361536204" Sep 15 07:23:26.374011 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:26.373934 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerStarted","Data":"5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee"} Sep 15 07:23:26.403000 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:26.402941 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.095154461 podStartE2EDuration="11.402918094s" totalImagesPullingTime="7.292447506s" totalInitContainerRuntime="15.316127ms" isStatefulPod=false podCreationTimestamp="2026-09-15 07:23:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:23:26.40169133 +0000 UTC m=+287.382853498" watchObservedRunningTime="2026-09-15 07:23:26.402918094 +0000 UTC m=+287.384080259" Sep 15 07:23:27.845076 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:27.845039 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:27.897833 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:27.897795 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:28.858255 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:28.858207 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:28.958811 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:28.958764 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:31.131478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:31.131432 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:31.195429 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:31.195384 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:33.877683 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.877651 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cc9f6ff85-9xrcm"] Sep 15 07:23:33.881438 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.881416 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:33.883920 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.883883 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 15 07:23:33.884287 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.884267 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 15 07:23:33.884928 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.884901 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4rwg4\"" Sep 15 07:23:33.884928 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.884927 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 15 07:23:33.885115 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.884998 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 15 07:23:33.885115 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.885000 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 15 07:23:33.892353 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.892331 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cc9f6ff85-9xrcm"] Sep 15 07:23:33.952233 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.952201 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93e73500-11fc-4138-94be-2f4007ed9058-console-oauth-config\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:33.952394 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.952254 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-service-ca\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:33.952394 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.952309 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-console-config\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:33.952394 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.952345 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93e73500-11fc-4138-94be-2f4007ed9058-console-serving-cert\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:33.952394 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.952375 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2vht\" (UniqueName: \"kubernetes.io/projected/93e73500-11fc-4138-94be-2f4007ed9058-kube-api-access-j2vht\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:33.952567 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:33.952476 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-oauth-serving-cert\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.053662 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.053625 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2vht\" (UniqueName: \"kubernetes.io/projected/93e73500-11fc-4138-94be-2f4007ed9058-kube-api-access-j2vht\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.053842 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.053714 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-oauth-serving-cert\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.053842 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.053763 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93e73500-11fc-4138-94be-2f4007ed9058-console-oauth-config\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.053842 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.053804 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-service-ca\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.053842 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.053836 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-console-config\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.054083 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.053861 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93e73500-11fc-4138-94be-2f4007ed9058-console-serving-cert\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.054637 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.054581 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-oauth-serving-cert\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.054772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.054658 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-console-config\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.054772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.054664 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-service-ca\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.056598 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.056554 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93e73500-11fc-4138-94be-2f4007ed9058-console-oauth-config\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.056717 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.056696 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93e73500-11fc-4138-94be-2f4007ed9058-console-serving-cert\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.065138 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.065099 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2vht\" (UniqueName: \"kubernetes.io/projected/93e73500-11fc-4138-94be-2f4007ed9058-kube-api-access-j2vht\") pod \"console-5cc9f6ff85-9xrcm\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.193823 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.193737 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:23:34.414924 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.414884 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:34.513477 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.513438 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:34.614362 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:34.614323 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:35.375937 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:35.375911 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bsrr6" Sep 15 07:23:37.404218 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:37.404089 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:37.461559 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:37.461519 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:37.910229 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:37.910195 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:37.972270 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:37.972213 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:38.354588 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:38.354540 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:38.407842 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:38.407806 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:40.944656 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:40.944624 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:23:40.945141 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:40.944781 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:23:40.955969 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:40.955944 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:23:40.956161 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:40.956144 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:23:40.962786 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:40.962762 2552 kubelet.go:1648] "Image garbage collection succeeded" Sep 15 07:23:41.179651 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:41.179570 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cc9f6ff85-9xrcm"] Sep 15 07:23:41.182770 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:23:41.182739 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e73500_11fc_4138_94be_2f4007ed9058.slice/crio-4a59c746f759d6fddc5fe5be4473211852552576b74c52294aebd1e6c3c318a2 WatchSource:0}: Error finding container 4a59c746f759d6fddc5fe5be4473211852552576b74c52294aebd1e6c3c318a2: Status 404 returned error can't find the container with id 4a59c746f759d6fddc5fe5be4473211852552576b74c52294aebd1e6c3c318a2 Sep 15 07:23:41.185358 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:41.185337 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 07:23:41.441374 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:41.441286 2552 generic.go:356] "Generic (PLEG): container finished" podID="449e530a-1a2b-4707-8272-48ec5851253d" containerID="29ab3197ded1e683ea410b60a4d10dc0d01ae4e4f1549c87d11af6db37f59d33" exitCode=0 Sep 15 07:23:41.441537 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:41.441376 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-46pvm" event={"ID":"449e530a-1a2b-4707-8272-48ec5851253d","Type":"ContainerDied","Data":"29ab3197ded1e683ea410b60a4d10dc0d01ae4e4f1549c87d11af6db37f59d33"} Sep 15 07:23:41.441839 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:41.441816 2552 scope.go:117] "RemoveContainer" containerID="29ab3197ded1e683ea410b60a4d10dc0d01ae4e4f1549c87d11af6db37f59d33" Sep 15 07:23:41.443089 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:41.443058 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cc9f6ff85-9xrcm" event={"ID":"93e73500-11fc-4138-94be-2f4007ed9058","Type":"ContainerStarted","Data":"4a59c746f759d6fddc5fe5be4473211852552576b74c52294aebd1e6c3c318a2"} Sep 15 07:23:41.444979 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:41.444955 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-q67fs" event={"ID":"e617ad16-f2c5-41a9-b78b-e2936a79f16b","Type":"ContainerStarted","Data":"22cf2dd53d3d6bee4dd8cf858593ac9eb550f340c1416294805e1a82b72f858f"} Sep 15 07:23:41.445230 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:41.445207 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-q67fs" Sep 15 07:23:41.457970 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:41.457946 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-q67fs" Sep 15 07:23:41.474707 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:41.474648 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-q67fs" podStartSLOduration=1.379205519 podStartE2EDuration="17.474626962s" totalImagesPullingTime="16.095421443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:23:41.474001457 +0000 UTC m=+302.455163625" watchObservedRunningTime="2026-09-15 07:23:41.474626962 +0000 UTC m=+302.455789132" Sep 15 07:23:41.556992 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:41.556943 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:41.614041 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:41.614000 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:42.454198 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:42.454150 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-46pvm" event={"ID":"449e530a-1a2b-4707-8272-48ec5851253d","Type":"ContainerStarted","Data":"073337bd5b16087e89448b0c7fc2f00fc524377958d10491663d2078b33ae68f"} Sep 15 07:23:44.373198 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:44.373152 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:44.436331 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:44.436270 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:45.473913 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:45.473822 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cc9f6ff85-9xrcm" event={"ID":"93e73500-11fc-4138-94be-2f4007ed9058","Type":"ContainerStarted","Data":"f0207543c8ce60b6681f9f4a95c0270d3364ee0bc4c54a8b18ceacb52b205230"} Sep 15 07:23:45.495580 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:45.495519 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cc9f6ff85-9xrcm" podStartSLOduration=8.609939756 podStartE2EDuration="12.495501725s" totalImagesPullingTime="3.885561969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:23:45.493688957 +0000 UTC m=+306.474851138" watchObservedRunningTime="2026-09-15 07:23:45.495501725 +0000 UTC m=+306.476663892" Sep 15 07:23:46.414515 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:46.414479 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:46.472138 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:46.472102 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:49.405690 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:49.405652 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:49.474888 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:49.474835 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:50.663348 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:50.663310 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:50.724709 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:50.724675 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:52.449474 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:52.449441 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cc9f6ff85-9xrcm"] Sep 15 07:23:52.555266 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:52.555225 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:52.656337 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:52.656304 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:52.719655 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:52.719564 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:52.943030 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:52.943000 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:52.995055 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:52.995024 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:23:54.194746 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:23:54.194718 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:24:04.405883 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:04.405816 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:24:04.473429 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:04.473398 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:24:17.469907 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.469855 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cc9f6ff85-9xrcm" podUID="93e73500-11fc-4138-94be-2f4007ed9058" containerName="console" containerID="cri-o://f0207543c8ce60b6681f9f4a95c0270d3364ee0bc4c54a8b18ceacb52b205230" gracePeriod=15 Sep 15 07:24:17.599845 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.599819 2552 generic.go:356] "Generic (PLEG): container finished" podID="93e73500-11fc-4138-94be-2f4007ed9058" containerID="f0207543c8ce60b6681f9f4a95c0270d3364ee0bc4c54a8b18ceacb52b205230" exitCode=0 Sep 15 07:24:17.599956 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.599892 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cc9f6ff85-9xrcm" event={"ID":"93e73500-11fc-4138-94be-2f4007ed9058","Type":"ContainerDied","Data":"f0207543c8ce60b6681f9f4a95c0270d3364ee0bc4c54a8b18ceacb52b205230"} Sep 15 07:24:17.748801 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.748779 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:24:17.821293 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.821266 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93e73500-11fc-4138-94be-2f4007ed9058-console-serving-cert\") pod \"93e73500-11fc-4138-94be-2f4007ed9058\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " Sep 15 07:24:17.821428 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.821321 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-service-ca\") pod \"93e73500-11fc-4138-94be-2f4007ed9058\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " Sep 15 07:24:17.821428 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.821347 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93e73500-11fc-4138-94be-2f4007ed9058-console-oauth-config\") pod \"93e73500-11fc-4138-94be-2f4007ed9058\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " Sep 15 07:24:17.821428 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.821372 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-oauth-serving-cert\") pod \"93e73500-11fc-4138-94be-2f4007ed9058\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " Sep 15 07:24:17.821428 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.821415 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-console-config\") pod \"93e73500-11fc-4138-94be-2f4007ed9058\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " Sep 15 07:24:17.821606 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.821442 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2vht\" (UniqueName: \"kubernetes.io/projected/93e73500-11fc-4138-94be-2f4007ed9058-kube-api-access-j2vht\") pod \"93e73500-11fc-4138-94be-2f4007ed9058\" (UID: \"93e73500-11fc-4138-94be-2f4007ed9058\") " Sep 15 07:24:17.821762 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.821736 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "93e73500-11fc-4138-94be-2f4007ed9058" (UID: "93e73500-11fc-4138-94be-2f4007ed9058"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:24:17.821837 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.821769 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-service-ca" (OuterVolumeSpecName: "service-ca") pod "93e73500-11fc-4138-94be-2f4007ed9058" (UID: "93e73500-11fc-4138-94be-2f4007ed9058"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:24:17.821837 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.821803 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-console-config" (OuterVolumeSpecName: "console-config") pod "93e73500-11fc-4138-94be-2f4007ed9058" (UID: "93e73500-11fc-4138-94be-2f4007ed9058"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:24:17.823594 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.823568 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93e73500-11fc-4138-94be-2f4007ed9058-kube-api-access-j2vht" (OuterVolumeSpecName: "kube-api-access-j2vht") pod "93e73500-11fc-4138-94be-2f4007ed9058" (UID: "93e73500-11fc-4138-94be-2f4007ed9058"). InnerVolumeSpecName "kube-api-access-j2vht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:24:17.823674 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.823598 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93e73500-11fc-4138-94be-2f4007ed9058-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "93e73500-11fc-4138-94be-2f4007ed9058" (UID: "93e73500-11fc-4138-94be-2f4007ed9058"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:24:17.823713 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.823680 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93e73500-11fc-4138-94be-2f4007ed9058-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "93e73500-11fc-4138-94be-2f4007ed9058" (UID: "93e73500-11fc-4138-94be-2f4007ed9058"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:24:17.922373 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.922336 2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93e73500-11fc-4138-94be-2f4007ed9058-console-serving-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:17.922373 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.922370 2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:17.922541 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.922383 2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93e73500-11fc-4138-94be-2f4007ed9058-console-oauth-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:17.922541 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.922395 2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-oauth-serving-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:17.922541 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.922406 2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93e73500-11fc-4138-94be-2f4007ed9058-console-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:17.922541 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:17.922417 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2vht\" (UniqueName: \"kubernetes.io/projected/93e73500-11fc-4138-94be-2f4007ed9058-kube-api-access-j2vht\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:18.605089 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:18.605051 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cc9f6ff85-9xrcm" event={"ID":"93e73500-11fc-4138-94be-2f4007ed9058","Type":"ContainerDied","Data":"4a59c746f759d6fddc5fe5be4473211852552576b74c52294aebd1e6c3c318a2"} Sep 15 07:24:18.605521 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:18.605121 2552 scope.go:117] "RemoveContainer" containerID="f0207543c8ce60b6681f9f4a95c0270d3364ee0bc4c54a8b18ceacb52b205230" Sep 15 07:24:18.605521 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:18.605066 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cc9f6ff85-9xrcm" Sep 15 07:24:18.627939 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:18.627913 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cc9f6ff85-9xrcm"] Sep 15 07:24:18.631963 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:18.631943 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cc9f6ff85-9xrcm"] Sep 15 07:24:19.473370 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:19.473340 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93e73500-11fc-4138-94be-2f4007ed9058" path="/var/lib/kubelet/pods/93e73500-11fc-4138-94be-2f4007ed9058/volumes" Sep 15 07:24:32.336291 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:32.336208 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:24:32.437512 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:32.437470 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:24:33.797361 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:33.797325 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:24:34.696912 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:34.696860 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:24:34.871842 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:34.871807 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:24:34.872250 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:34.872109 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="alertmanager" containerID="cri-o://66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763" gracePeriod=120 Sep 15 07:24:34.872250 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:34.872187 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="kube-rbac-proxy-metric" containerID="cri-o://c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d" gracePeriod=120 Sep 15 07:24:34.872370 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:34.872231 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="kube-rbac-proxy-web" containerID="cri-o://140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099" gracePeriod=120 Sep 15 07:24:34.872370 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:34.872264 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="config-reloader" containerID="cri-o://2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c" gracePeriod=120 Sep 15 07:24:34.872370 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:34.872282 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="kube-rbac-proxy" containerID="cri-o://1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b" gracePeriod=120 Sep 15 07:24:34.872523 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:34.872221 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="prom-label-proxy" containerID="cri-o://5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee" gracePeriod=120 Sep 15 07:24:35.670932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:35.670900 2552 generic.go:356] "Generic (PLEG): container finished" podID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerID="5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee" exitCode=0 Sep 15 07:24:35.670932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:35.670924 2552 generic.go:356] "Generic (PLEG): container finished" podID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerID="c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d" exitCode=0 Sep 15 07:24:35.670932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:35.670930 2552 generic.go:356] "Generic (PLEG): container finished" podID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerID="1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b" exitCode=0 Sep 15 07:24:35.670932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:35.670936 2552 generic.go:356] "Generic (PLEG): container finished" podID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerID="2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c" exitCode=0 Sep 15 07:24:35.670932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:35.670942 2552 generic.go:356] "Generic (PLEG): container finished" podID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerID="66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763" exitCode=0 Sep 15 07:24:35.671212 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:35.670975 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerDied","Data":"5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee"} Sep 15 07:24:35.671212 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:35.671039 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerDied","Data":"c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d"} Sep 15 07:24:35.671212 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:35.671066 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerDied","Data":"1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b"} Sep 15 07:24:35.671212 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:35.671090 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerDied","Data":"2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c"} Sep 15 07:24:35.671212 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:35.671113 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerDied","Data":"66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763"} Sep 15 07:24:36.116119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.116098 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.154372 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154349 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-main-tls\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154467 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154390 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0491b87-e447-4e42-aa9d-281631cd6aab-config-out\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154467 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154436 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5nfl\" (UniqueName: \"kubernetes.io/projected/c0491b87-e447-4e42-aa9d-281631cd6aab-kube-api-access-n5nfl\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154556 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154469 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-config-volume\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154556 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154509 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-main-db\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154556 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154532 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0491b87-e447-4e42-aa9d-281631cd6aab-tls-assets\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154703 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154561 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy-web\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154703 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154588 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-cluster-tls-config\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154703 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154613 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-metrics-client-ca\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154703 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154640 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154703 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154671 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-trusted-ca-bundle\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154974 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154718 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154974 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154743 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-web-config\") pod \"c0491b87-e447-4e42-aa9d-281631cd6aab\" (UID: \"c0491b87-e447-4e42-aa9d-281631cd6aab\") " Sep 15 07:24:36.154974 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.154922 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:24:36.155569 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.155523 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:24:36.155569 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.155548 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:24:36.157355 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.157330 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0491b87-e447-4e42-aa9d-281631cd6aab-config-out" (OuterVolumeSpecName: "config-out") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:24:36.157509 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.157484 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0491b87-e447-4e42-aa9d-281631cd6aab-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:24:36.157570 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.157501 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0491b87-e447-4e42-aa9d-281631cd6aab-kube-api-access-n5nfl" (OuterVolumeSpecName: "kube-api-access-n5nfl") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "kube-api-access-n5nfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:24:36.157750 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.157673 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:24:36.157750 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.157715 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:24:36.157976 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.157952 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:24:36.158180 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.158154 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:24:36.159245 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.159201 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-config-volume" (OuterVolumeSpecName: "config-volume") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:24:36.161674 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.161654 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:24:36.167454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.167433 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-web-config" (OuterVolumeSpecName: "web-config") pod "c0491b87-e447-4e42-aa9d-281631cd6aab" (UID: "c0491b87-e447-4e42-aa9d-281631cd6aab"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:24:36.255389 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255355 2552 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-main-db\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.255534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255404 2552 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0491b87-e447-4e42-aa9d-281631cd6aab-tls-assets\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.255534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255420 2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.255534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255429 2552 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-cluster-tls-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.255534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255438 2552 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-metrics-client-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.255534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255446 2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.255534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255455 2552 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0491b87-e447-4e42-aa9d-281631cd6aab-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.255534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255465 2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.255534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255473 2552 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-web-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.255534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255482 2552 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-secret-alertmanager-main-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.255534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255491 2552 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0491b87-e447-4e42-aa9d-281631cd6aab-config-out\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.255534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255499 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5nfl\" (UniqueName: \"kubernetes.io/projected/c0491b87-e447-4e42-aa9d-281631cd6aab-kube-api-access-n5nfl\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.255534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.255506 2552 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0491b87-e447-4e42-aa9d-281631cd6aab-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:24:36.677507 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.677432 2552 generic.go:356] "Generic (PLEG): container finished" podID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerID="140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099" exitCode=0 Sep 15 07:24:36.677507 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.677487 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerDied","Data":"140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099"} Sep 15 07:24:36.677671 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.677509 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.677671 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.677542 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0491b87-e447-4e42-aa9d-281631cd6aab","Type":"ContainerDied","Data":"2a32beedb37aa0a229f44e4576707ca2ba2c96e5df5945c96104af45158124d4"} Sep 15 07:24:36.677671 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.677576 2552 scope.go:117] "RemoveContainer" containerID="5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee" Sep 15 07:24:36.685915 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.685898 2552 scope.go:117] "RemoveContainer" containerID="c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d" Sep 15 07:24:36.693039 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.693024 2552 scope.go:117] "RemoveContainer" containerID="1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b" Sep 15 07:24:36.699160 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.699142 2552 scope.go:117] "RemoveContainer" containerID="140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099" Sep 15 07:24:36.705534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.705515 2552 scope.go:117] "RemoveContainer" containerID="2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c" Sep 15 07:24:36.710778 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.709099 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:24:36.714291 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.714269 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:24:36.717426 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.717409 2552 scope.go:117] "RemoveContainer" containerID="66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763" Sep 15 07:24:36.724013 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.723996 2552 scope.go:117] "RemoveContainer" containerID="a47c3b61a885ac5095539fc249510388181dcebe470c30d55f894b0dd6911cc9" Sep 15 07:24:36.730197 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.730182 2552 scope.go:117] "RemoveContainer" containerID="5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee" Sep 15 07:24:36.730478 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:24:36.730456 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee\": container with ID starting with 5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee not found: ID does not exist" containerID="5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee" Sep 15 07:24:36.730528 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.730485 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee"} err="failed to get container status \"5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee\": rpc error: code = NotFound desc = could not find container \"5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee\": container with ID starting with 5b9847e78466d16753c001b7c58f852b0e569f8d0b56e95c67745f99f4ef30ee not found: ID does not exist" Sep 15 07:24:36.730528 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.730524 2552 scope.go:117] "RemoveContainer" containerID="c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d" Sep 15 07:24:36.730772 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:24:36.730758 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d\": container with ID starting with c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d not found: ID does not exist" containerID="c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d" Sep 15 07:24:36.730831 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.730773 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d"} err="failed to get container status \"c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d\": rpc error: code = NotFound desc = could not find container \"c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d\": container with ID starting with c6c61825fff4f9b938c4cc24f96f6f132fc51aea42e1a3d9dc874e34754b351d not found: ID does not exist" Sep 15 07:24:36.730831 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.730784 2552 scope.go:117] "RemoveContainer" containerID="1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b" Sep 15 07:24:36.731022 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:24:36.731003 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b\": container with ID starting with 1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b not found: ID does not exist" containerID="1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b" Sep 15 07:24:36.731067 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.731028 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b"} err="failed to get container status \"1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b\": rpc error: code = NotFound desc = could not find container \"1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b\": container with ID starting with 1f3e7c9247165a553a3300ccbdcc2edf033a5bf633f05593f0cfb63dc9e8cb1b not found: ID does not exist" Sep 15 07:24:36.731067 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.731043 2552 scope.go:117] "RemoveContainer" containerID="140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099" Sep 15 07:24:36.731301 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:24:36.731283 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099\": container with ID starting with 140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099 not found: ID does not exist" containerID="140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099" Sep 15 07:24:36.731368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.731317 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099"} err="failed to get container status \"140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099\": rpc error: code = NotFound desc = could not find container \"140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099\": container with ID starting with 140b48c4ba7a6bcc6897534da72472c5e5da1d945a75ef919f1c960570bf6099 not found: ID does not exist" Sep 15 07:24:36.731368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.731338 2552 scope.go:117] "RemoveContainer" containerID="2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c" Sep 15 07:24:36.731593 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:24:36.731577 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c\": container with ID starting with 2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c not found: ID does not exist" containerID="2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c" Sep 15 07:24:36.731650 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.731597 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c"} err="failed to get container status \"2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c\": rpc error: code = NotFound desc = could not find container \"2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c\": container with ID starting with 2a5937d0f86a29ec1fb2d093e170682d20c14272afaae704938c533e37ccd68c not found: ID does not exist" Sep 15 07:24:36.731650 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.731611 2552 scope.go:117] "RemoveContainer" containerID="66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763" Sep 15 07:24:36.731813 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:24:36.731798 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763\": container with ID starting with 66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763 not found: ID does not exist" containerID="66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763" Sep 15 07:24:36.731855 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.731816 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763"} err="failed to get container status \"66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763\": rpc error: code = NotFound desc = could not find container \"66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763\": container with ID starting with 66f438fedc4bdf7faee031c5367e57bb877608d8e8f2e4eada71f1a85ef77763 not found: ID does not exist" Sep 15 07:24:36.731855 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.731829 2552 scope.go:117] "RemoveContainer" containerID="a47c3b61a885ac5095539fc249510388181dcebe470c30d55f894b0dd6911cc9" Sep 15 07:24:36.732054 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:24:36.732036 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a47c3b61a885ac5095539fc249510388181dcebe470c30d55f894b0dd6911cc9\": container with ID starting with a47c3b61a885ac5095539fc249510388181dcebe470c30d55f894b0dd6911cc9 not found: ID does not exist" containerID="a47c3b61a885ac5095539fc249510388181dcebe470c30d55f894b0dd6911cc9" Sep 15 07:24:36.732098 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.732059 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a47c3b61a885ac5095539fc249510388181dcebe470c30d55f894b0dd6911cc9"} err="failed to get container status \"a47c3b61a885ac5095539fc249510388181dcebe470c30d55f894b0dd6911cc9\": rpc error: code = NotFound desc = could not find container \"a47c3b61a885ac5095539fc249510388181dcebe470c30d55f894b0dd6911cc9\": container with ID starting with a47c3b61a885ac5095539fc249510388181dcebe470c30d55f894b0dd6911cc9 not found: ID does not exist" Sep 15 07:24:36.753682 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.753662 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:24:36.753996 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.753981 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="alertmanager" Sep 15 07:24:36.754078 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.753999 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="alertmanager" Sep 15 07:24:36.754078 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754011 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="config-reloader" Sep 15 07:24:36.754078 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754021 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="config-reloader" Sep 15 07:24:36.754078 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754039 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="kube-rbac-proxy-web" Sep 15 07:24:36.754078 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754048 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="kube-rbac-proxy-web" Sep 15 07:24:36.754078 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754064 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="kube-rbac-proxy" Sep 15 07:24:36.754078 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754072 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="kube-rbac-proxy" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754083 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="init-config-reloader" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754092 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="init-config-reloader" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754103 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="prom-label-proxy" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754112 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="prom-label-proxy" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754130 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="kube-rbac-proxy-metric" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754138 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="kube-rbac-proxy-metric" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754147 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93e73500-11fc-4138-94be-2f4007ed9058" containerName="console" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754154 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="93e73500-11fc-4138-94be-2f4007ed9058" containerName="console" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754219 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="kube-rbac-proxy-web" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754232 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="kube-rbac-proxy" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754243 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="kube-rbac-proxy-metric" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754251 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="alertmanager" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754260 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="93e73500-11fc-4138-94be-2f4007ed9058" containerName="console" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754269 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="prom-label-proxy" Sep 15 07:24:36.754459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.754278 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" containerName="config-reloader" Sep 15 07:24:36.758341 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.758323 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.761431 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.761408 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 15 07:24:36.761513 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.761484 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 15 07:24:36.761564 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.761534 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 15 07:24:36.761604 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.761580 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2jf6v\"" Sep 15 07:24:36.761748 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.761731 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 15 07:24:36.761813 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.761800 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 15 07:24:36.762172 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.762152 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 15 07:24:36.762640 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.762626 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 15 07:24:36.762709 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.762695 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 15 07:24:36.770856 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.770836 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 15 07:24:36.782847 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.782827 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:24:36.859596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859564 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.859596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859600 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c377b38a-a1ed-4a58-a11d-7ca077287976-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.859791 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859641 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbwk4\" (UniqueName: \"kubernetes.io/projected/c377b38a-a1ed-4a58-a11d-7ca077287976-kube-api-access-mbwk4\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.859791 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859685 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.859791 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859704 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.859791 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859719 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c377b38a-a1ed-4a58-a11d-7ca077287976-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.859791 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859741 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c377b38a-a1ed-4a58-a11d-7ca077287976-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.859791 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859756 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.859791 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859777 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-web-config\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.859791 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859791 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c377b38a-a1ed-4a58-a11d-7ca077287976-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.860073 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859857 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.860073 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859887 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c377b38a-a1ed-4a58-a11d-7ca077287976-config-out\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.860073 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.859905 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-config-volume\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.960916 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.960827 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c377b38a-a1ed-4a58-a11d-7ca077287976-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.960916 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.960860 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.960916 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.960903 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-web-config\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.961116 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.960924 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c377b38a-a1ed-4a58-a11d-7ca077287976-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.961116 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.960988 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.961116 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.961016 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c377b38a-a1ed-4a58-a11d-7ca077287976-config-out\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.961116 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.961042 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-config-volume\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.961116 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.961067 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.961116 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.961093 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c377b38a-a1ed-4a58-a11d-7ca077287976-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.961116 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.961118 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbwk4\" (UniqueName: \"kubernetes.io/projected/c377b38a-a1ed-4a58-a11d-7ca077287976-kube-api-access-mbwk4\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.962143 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.961658 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.962143 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.961710 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c377b38a-a1ed-4a58-a11d-7ca077287976-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.962143 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.961712 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.962143 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.961761 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c377b38a-a1ed-4a58-a11d-7ca077287976-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.962775 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.962208 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c377b38a-a1ed-4a58-a11d-7ca077287976-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.962853 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.962800 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c377b38a-a1ed-4a58-a11d-7ca077287976-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.964055 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.964033 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c377b38a-a1ed-4a58-a11d-7ca077287976-config-out\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.964150 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.964036 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-web-config\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.964803 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.964758 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.964914 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.964807 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.964914 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.964811 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c377b38a-a1ed-4a58-a11d-7ca077287976-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.964989 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.964910 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-config-volume\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.965148 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.965130 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.965355 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.965335 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.965535 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.965518 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c377b38a-a1ed-4a58-a11d-7ca077287976-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:36.970131 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:36.970115 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbwk4\" (UniqueName: \"kubernetes.io/projected/c377b38a-a1ed-4a58-a11d-7ca077287976-kube-api-access-mbwk4\") pod \"alertmanager-main-0\" (UID: \"c377b38a-a1ed-4a58-a11d-7ca077287976\") " pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:37.067562 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:37.067540 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 15 07:24:37.346134 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:37.346107 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 15 07:24:37.348618 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:24:37.348593 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc377b38a_a1ed_4a58_a11d_7ca077287976.slice/crio-5b075acfa4efb03f4b430f9ed65d152c8a351bdf8c0383f06370372571b82759 WatchSource:0}: Error finding container 5b075acfa4efb03f4b430f9ed65d152c8a351bdf8c0383f06370372571b82759: Status 404 returned error can't find the container with id 5b075acfa4efb03f4b430f9ed65d152c8a351bdf8c0383f06370372571b82759 Sep 15 07:24:37.473299 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:37.473262 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0491b87-e447-4e42-aa9d-281631cd6aab" path="/var/lib/kubelet/pods/c0491b87-e447-4e42-aa9d-281631cd6aab/volumes" Sep 15 07:24:37.682952 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:37.682860 2552 generic.go:356] "Generic (PLEG): container finished" podID="c377b38a-a1ed-4a58-a11d-7ca077287976" containerID="e5330062a0af1c8af01e4502a97cf348f70f0dd16fd5405d69dd58cc54c92c86" exitCode=0 Sep 15 07:24:37.683227 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:37.682969 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c377b38a-a1ed-4a58-a11d-7ca077287976","Type":"ContainerDied","Data":"e5330062a0af1c8af01e4502a97cf348f70f0dd16fd5405d69dd58cc54c92c86"} Sep 15 07:24:37.683227 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:37.683041 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c377b38a-a1ed-4a58-a11d-7ca077287976","Type":"ContainerStarted","Data":"5b075acfa4efb03f4b430f9ed65d152c8a351bdf8c0383f06370372571b82759"} Sep 15 07:24:38.695862 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.695826 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c377b38a-a1ed-4a58-a11d-7ca077287976","Type":"ContainerStarted","Data":"f4da8c09fd3c64a12e021a69677392f7a03a7a23e6b364d991074911cf2781e2"} Sep 15 07:24:38.696280 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.695937 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c377b38a-a1ed-4a58-a11d-7ca077287976","Type":"ContainerStarted","Data":"42ad0f4e71fa279257459a5c40b24574e8ca1cba7522ef00d88454e78fb4e719"} Sep 15 07:24:38.696280 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.695965 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c377b38a-a1ed-4a58-a11d-7ca077287976","Type":"ContainerStarted","Data":"402c214442b5ab9ffd27b8b371d5c8db23b72bd6fba59493b0006f397169a6b3"} Sep 15 07:24:38.696280 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.695989 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c377b38a-a1ed-4a58-a11d-7ca077287976","Type":"ContainerStarted","Data":"69ce72b82b3c816e15c4edc068622ace08e8ad2dd33a6c6187524d4c80c5cecd"} Sep 15 07:24:38.696280 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.696011 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c377b38a-a1ed-4a58-a11d-7ca077287976","Type":"ContainerStarted","Data":"6f5c6749a2987aa8134daa708247462813d48940168693f4c5ef69b00784d7f3"} Sep 15 07:24:38.696280 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.696033 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c377b38a-a1ed-4a58-a11d-7ca077287976","Type":"ContainerStarted","Data":"979b7b27f878cdb9d4c4480560acb6d3f47488c49fcdd9ca29e8d61e9965f3fd"} Sep 15 07:24:38.723051 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.723005 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.709386952 podStartE2EDuration="2.722986835s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.599883ms" isStatefulPod=false podCreationTimestamp="2026-09-15 07:24:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:24:38.722449857 +0000 UTC m=+359.703612024" watchObservedRunningTime="2026-09-15 07:24:38.722986835 +0000 UTC m=+359.704149002" Sep 15 07:24:38.883931 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.883891 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl"] Sep 15 07:24:38.886663 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.886643 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:38.889933 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.889901 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 15 07:24:38.890176 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.889927 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 15 07:24:38.890285 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.889927 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wd6hm\"" Sep 15 07:24:38.890285 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.890217 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 15 07:24:38.890285 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.889992 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 15 07:24:38.890487 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.890370 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 15 07:24:38.895813 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.895794 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 15 07:24:38.902903 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.902879 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl"] Sep 15 07:24:38.977106 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.977072 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/119ff648-9dee-42b1-b951-864f117fad51-federate-client-tls\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:38.977243 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.977111 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/119ff648-9dee-42b1-b951-864f117fad51-secret-telemeter-client\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:38.977243 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.977164 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/119ff648-9dee-42b1-b951-864f117fad51-serving-certs-ca-bundle\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:38.977243 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.977218 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/119ff648-9dee-42b1-b951-864f117fad51-telemeter-client-tls\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:38.977364 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.977242 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/119ff648-9dee-42b1-b951-864f117fad51-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:38.977364 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.977261 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/119ff648-9dee-42b1-b951-864f117fad51-metrics-client-ca\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:38.977364 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.977313 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/119ff648-9dee-42b1-b951-864f117fad51-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:38.977364 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:38.977342 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbzx7\" (UniqueName: \"kubernetes.io/projected/119ff648-9dee-42b1-b951-864f117fad51-kube-api-access-tbzx7\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.078633 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.078599 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/119ff648-9dee-42b1-b951-864f117fad51-federate-client-tls\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.078633 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.078636 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/119ff648-9dee-42b1-b951-864f117fad51-secret-telemeter-client\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.078909 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.078658 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/119ff648-9dee-42b1-b951-864f117fad51-serving-certs-ca-bundle\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.078909 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.078684 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/119ff648-9dee-42b1-b951-864f117fad51-telemeter-client-tls\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.078909 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.078714 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/119ff648-9dee-42b1-b951-864f117fad51-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.078909 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.078739 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/119ff648-9dee-42b1-b951-864f117fad51-metrics-client-ca\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.078909 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.078782 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/119ff648-9dee-42b1-b951-864f117fad51-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.078909 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.078812 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbzx7\" (UniqueName: \"kubernetes.io/projected/119ff648-9dee-42b1-b951-864f117fad51-kube-api-access-tbzx7\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.079557 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.079528 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/119ff648-9dee-42b1-b951-864f117fad51-serving-certs-ca-bundle\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.079716 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.079691 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/119ff648-9dee-42b1-b951-864f117fad51-metrics-client-ca\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.079846 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.079827 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/119ff648-9dee-42b1-b951-864f117fad51-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.081535 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.081510 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/119ff648-9dee-42b1-b951-864f117fad51-telemeter-client-tls\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.081657 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.081535 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/119ff648-9dee-42b1-b951-864f117fad51-federate-client-tls\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.081767 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.081744 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/119ff648-9dee-42b1-b951-864f117fad51-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.081826 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.081803 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/119ff648-9dee-42b1-b951-864f117fad51-secret-telemeter-client\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.086273 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.086244 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbzx7\" (UniqueName: \"kubernetes.io/projected/119ff648-9dee-42b1-b951-864f117fad51-kube-api-access-tbzx7\") pod \"telemeter-client-6cf9c55f4d-sklkl\" (UID: \"119ff648-9dee-42b1-b951-864f117fad51\") " pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.197781 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.197744 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" Sep 15 07:24:39.486744 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.486659 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl"] Sep 15 07:24:39.488077 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:24:39.488046 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod119ff648_9dee_42b1_b951_864f117fad51.slice/crio-cf4b6430c1e5903202c55b36e868a9de319fefe4a805f4de680571dc0c4ea93b WatchSource:0}: Error finding container cf4b6430c1e5903202c55b36e868a9de319fefe4a805f4de680571dc0c4ea93b: Status 404 returned error can't find the container with id cf4b6430c1e5903202c55b36e868a9de319fefe4a805f4de680571dc0c4ea93b Sep 15 07:24:39.700857 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:39.700771 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" event={"ID":"119ff648-9dee-42b1-b951-864f117fad51","Type":"ContainerStarted","Data":"cf4b6430c1e5903202c55b36e868a9de319fefe4a805f4de680571dc0c4ea93b"} Sep 15 07:24:41.710939 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.710903 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" event={"ID":"119ff648-9dee-42b1-b951-864f117fad51","Type":"ContainerStarted","Data":"80f298ba0bc02dbd494a7f4c749caca73ffd7c375afc1644b2b191f7c8898df6"} Sep 15 07:24:41.711295 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.710958 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" event={"ID":"119ff648-9dee-42b1-b951-864f117fad51","Type":"ContainerStarted","Data":"1e3a77c03110537c512a281a06679a0cc48a91a838114fc8bf0b5b86bd0c73d8"} Sep 15 07:24:41.711295 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.710978 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" event={"ID":"119ff648-9dee-42b1-b951-864f117fad51","Type":"ContainerStarted","Data":"15cdb9f364abd2d60384114db8a059da5b84cfb1aadd9fc7c37203e4a8ad6893"} Sep 15 07:24:41.735109 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.735063 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6cf9c55f4d-sklkl" podStartSLOduration=2.060702302 podStartE2EDuration="3.735047369s" totalImagesPullingTime="1.674345067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:24:41.732829571 +0000 UTC m=+362.713991737" watchObservedRunningTime="2026-09-15 07:24:41.735047369 +0000 UTC m=+362.716209562" Sep 15 07:24:41.818648 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.818614 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-697b678444-96gjn"] Sep 15 07:24:41.820835 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.820813 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:41.823365 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.823343 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 15 07:24:41.823495 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.823477 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 15 07:24:41.823563 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.823528 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 15 07:24:41.823779 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.823753 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 15 07:24:41.823882 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.823846 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 15 07:24:41.824009 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.823992 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4rwg4\"" Sep 15 07:24:41.829332 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.829308 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 15 07:24:41.833745 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.833712 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-697b678444-96gjn"] Sep 15 07:24:41.881369 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.881336 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:24:41.904450 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.904418 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-oauth-config\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:41.904583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.904497 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-serving-cert\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:41.904583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.904523 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-service-ca\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:41.904583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.904547 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpj4h\" (UniqueName: \"kubernetes.io/projected/16fb10e4-0872-4f43-adfe-d113e696b8bd-kube-api-access-kpj4h\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:41.904583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.904570 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-oauth-serving-cert\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:41.904797 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.904595 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-config\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:41.904797 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.904643 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-trusted-ca-bundle\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:41.991455 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:41.991408 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:24:42.005345 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.005317 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-serving-cert\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.005497 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.005350 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-service-ca\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.005497 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.005367 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpj4h\" (UniqueName: \"kubernetes.io/projected/16fb10e4-0872-4f43-adfe-d113e696b8bd-kube-api-access-kpj4h\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.005497 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.005385 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-oauth-serving-cert\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.005615 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.005597 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-config\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.005686 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.005673 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-trusted-ca-bundle\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.005742 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.005729 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-oauth-config\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.006237 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.006208 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-service-ca\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.006355 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.006208 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-oauth-serving-cert\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.006701 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.006681 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-config\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.006826 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.006799 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-trusted-ca-bundle\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.008519 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.008477 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-oauth-config\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.008692 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.008670 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-serving-cert\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.014849 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.014830 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpj4h\" (UniqueName: \"kubernetes.io/projected/16fb10e4-0872-4f43-adfe-d113e696b8bd-kube-api-access-kpj4h\") pod \"console-697b678444-96gjn\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.132088 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.132042 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:42.419785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.419755 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-697b678444-96gjn"] Sep 15 07:24:42.420559 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:24:42.420529 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16fb10e4_0872_4f43_adfe_d113e696b8bd.slice/crio-b69302ab1ec3b416cea0ac138167870dda5184126378b3b419703a8e86caf8e2 WatchSource:0}: Error finding container b69302ab1ec3b416cea0ac138167870dda5184126378b3b419703a8e86caf8e2: Status 404 returned error can't find the container with id b69302ab1ec3b416cea0ac138167870dda5184126378b3b419703a8e86caf8e2 Sep 15 07:24:42.716064 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.715972 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697b678444-96gjn" event={"ID":"16fb10e4-0872-4f43-adfe-d113e696b8bd","Type":"ContainerStarted","Data":"b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d"} Sep 15 07:24:42.716064 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.716032 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697b678444-96gjn" event={"ID":"16fb10e4-0872-4f43-adfe-d113e696b8bd","Type":"ContainerStarted","Data":"b69302ab1ec3b416cea0ac138167870dda5184126378b3b419703a8e86caf8e2"} Sep 15 07:24:42.734593 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:42.734542 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-697b678444-96gjn" podStartSLOduration=1.734526968 podStartE2EDuration="1.734526968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:24:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:24:42.732939492 +0000 UTC m=+363.714101659" watchObservedRunningTime="2026-09-15 07:24:42.734526968 +0000 UTC m=+363.715689135" Sep 15 07:24:48.682367 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:48.682326 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:24:48.742916 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:48.742892 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:24:52.132606 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:52.132561 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:52.133039 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:52.132630 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:52.137087 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:52.137067 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:52.756467 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:52.756440 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-697b678444-96gjn" Sep 15 07:24:52.900816 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:52.900787 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:24:52.957152 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:52.957122 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:24:53.008128 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:24:53.008055 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:25:12.104437 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:12.104399 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:25:12.156207 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:12.156169 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:25:49.039347 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:49.039307 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:25:49.097083 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:49.097049 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:25:55.869310 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:55.869276 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b965b86cb-7ntcg"] Sep 15 07:25:55.871927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:55.871891 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:55.883081 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:55.883057 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b965b86cb-7ntcg"] Sep 15 07:25:55.925176 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:55.925144 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:25:56.001403 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.001373 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-service-ca\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.001569 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.001415 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-oauth-serving-cert\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.001569 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.001445 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-serving-cert\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.001569 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.001500 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-trusted-ca-bundle\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.001569 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.001532 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-oauth-config\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.001569 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.001545 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-config\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.001569 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.001559 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxpzp\" (UniqueName: \"kubernetes.io/projected/fc7ead54-7930-4550-9baa-990d86fdf2ce-kube-api-access-wxpzp\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.004897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.004854 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:25:56.102186 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.102154 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-trusted-ca-bundle\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.102363 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.102204 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-oauth-config\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.102363 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.102220 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-config\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.102363 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.102235 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxpzp\" (UniqueName: \"kubernetes.io/projected/fc7ead54-7930-4550-9baa-990d86fdf2ce-kube-api-access-wxpzp\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.102363 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.102259 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-service-ca\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.102363 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.102286 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-oauth-serving-cert\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.102363 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.102305 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-serving-cert\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.103018 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.102991 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-config\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.103157 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.103121 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-service-ca\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.103157 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.103119 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-oauth-serving-cert\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.103347 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.103325 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-trusted-ca-bundle\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.104819 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.104798 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-oauth-config\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.104948 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.104931 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-serving-cert\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.109839 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.109820 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxpzp\" (UniqueName: \"kubernetes.io/projected/fc7ead54-7930-4550-9baa-990d86fdf2ce-kube-api-access-wxpzp\") pod \"console-7b965b86cb-7ntcg\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.181543 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.181491 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:25:56.450635 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.450610 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b965b86cb-7ntcg"] Sep 15 07:25:56.453108 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:25:56.453077 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc7ead54_7930_4550_9baa_990d86fdf2ce.slice/crio-645faa2110fde24773afad02cc2a740ff6f5f774e2d68ccd9839000f94a9024f WatchSource:0}: Error finding container 645faa2110fde24773afad02cc2a740ff6f5f774e2d68ccd9839000f94a9024f: Status 404 returned error can't find the container with id 645faa2110fde24773afad02cc2a740ff6f5f774e2d68ccd9839000f94a9024f Sep 15 07:25:56.998447 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.998413 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b965b86cb-7ntcg" event={"ID":"fc7ead54-7930-4550-9baa-990d86fdf2ce","Type":"ContainerStarted","Data":"251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201"} Sep 15 07:25:56.998821 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:56.998463 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b965b86cb-7ntcg" event={"ID":"fc7ead54-7930-4550-9baa-990d86fdf2ce","Type":"ContainerStarted","Data":"645faa2110fde24773afad02cc2a740ff6f5f774e2d68ccd9839000f94a9024f"} Sep 15 07:25:57.018017 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:25:57.017973 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b965b86cb-7ntcg" podStartSLOduration=2.017959291 podStartE2EDuration="2.017959291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:25:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:25:57.015984069 +0000 UTC m=+437.997146235" watchObservedRunningTime="2026-09-15 07:25:57.017959291 +0000 UTC m=+437.999121457" Sep 15 07:26:01.059803 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:01.059767 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:26:01.231292 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:01.231245 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:26:04.410533 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:04.410469 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:26:04.511189 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:04.511155 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:26:04.872742 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:04.872710 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:26:06.181969 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:06.181919 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:26:06.181969 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:06.181986 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:26:06.186823 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:06.186799 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:26:07.039847 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:07.039818 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:26:07.090135 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:07.090106 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-697b678444-96gjn"] Sep 15 07:26:07.205119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:07.205087 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:26:07.306013 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:07.305934 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:26:07.359885 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:07.359841 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:26:32.115222 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.115181 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-697b678444-96gjn" podUID="16fb10e4-0872-4f43-adfe-d113e696b8bd" containerName="console" containerID="cri-o://b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d" gracePeriod=15 Sep 15 07:26:32.357115 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.357092 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697b678444-96gjn" Sep 15 07:26:32.470017 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.469939 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-serving-cert\") pod \"16fb10e4-0872-4f43-adfe-d113e696b8bd\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " Sep 15 07:26:32.470017 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.469992 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-oauth-serving-cert\") pod \"16fb10e4-0872-4f43-adfe-d113e696b8bd\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " Sep 15 07:26:32.470238 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.470044 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-trusted-ca-bundle\") pod \"16fb10e4-0872-4f43-adfe-d113e696b8bd\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " Sep 15 07:26:32.470238 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.470081 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-service-ca\") pod \"16fb10e4-0872-4f43-adfe-d113e696b8bd\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " Sep 15 07:26:32.470238 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.470202 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-oauth-config\") pod \"16fb10e4-0872-4f43-adfe-d113e696b8bd\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " Sep 15 07:26:32.470394 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.470252 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-config\") pod \"16fb10e4-0872-4f43-adfe-d113e696b8bd\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " Sep 15 07:26:32.470394 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.470307 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpj4h\" (UniqueName: \"kubernetes.io/projected/16fb10e4-0872-4f43-adfe-d113e696b8bd-kube-api-access-kpj4h\") pod \"16fb10e4-0872-4f43-adfe-d113e696b8bd\" (UID: \"16fb10e4-0872-4f43-adfe-d113e696b8bd\") " Sep 15 07:26:32.470499 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.470414 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "16fb10e4-0872-4f43-adfe-d113e696b8bd" (UID: "16fb10e4-0872-4f43-adfe-d113e696b8bd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:26:32.470499 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.470434 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-service-ca" (OuterVolumeSpecName: "service-ca") pod "16fb10e4-0872-4f43-adfe-d113e696b8bd" (UID: "16fb10e4-0872-4f43-adfe-d113e696b8bd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:26:32.470499 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.470453 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "16fb10e4-0872-4f43-adfe-d113e696b8bd" (UID: "16fb10e4-0872-4f43-adfe-d113e696b8bd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:26:32.470686 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.470659 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-config" (OuterVolumeSpecName: "console-config") pod "16fb10e4-0872-4f43-adfe-d113e696b8bd" (UID: "16fb10e4-0872-4f43-adfe-d113e696b8bd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:26:32.470748 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.470671 2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:26:32.470748 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.470712 2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-oauth-serving-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:26:32.470748 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.470729 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-trusted-ca-bundle\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:26:32.472152 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.472125 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "16fb10e4-0872-4f43-adfe-d113e696b8bd" (UID: "16fb10e4-0872-4f43-adfe-d113e696b8bd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:26:32.472247 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.472152 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "16fb10e4-0872-4f43-adfe-d113e696b8bd" (UID: "16fb10e4-0872-4f43-adfe-d113e696b8bd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:26:32.472298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.472237 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16fb10e4-0872-4f43-adfe-d113e696b8bd-kube-api-access-kpj4h" (OuterVolumeSpecName: "kube-api-access-kpj4h") pod "16fb10e4-0872-4f43-adfe-d113e696b8bd" (UID: "16fb10e4-0872-4f43-adfe-d113e696b8bd"). InnerVolumeSpecName "kube-api-access-kpj4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:26:32.571788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.571758 2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-oauth-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:26:32.571788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.571779 2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:26:32.571788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.571789 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpj4h\" (UniqueName: \"kubernetes.io/projected/16fb10e4-0872-4f43-adfe-d113e696b8bd-kube-api-access-kpj4h\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:26:32.572093 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:32.571798 2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16fb10e4-0872-4f43-adfe-d113e696b8bd-console-serving-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:26:33.133250 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:33.133216 2552 generic.go:356] "Generic (PLEG): container finished" podID="16fb10e4-0872-4f43-adfe-d113e696b8bd" containerID="b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d" exitCode=0 Sep 15 07:26:33.133703 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:33.133261 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697b678444-96gjn" Sep 15 07:26:33.133703 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:33.133306 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697b678444-96gjn" event={"ID":"16fb10e4-0872-4f43-adfe-d113e696b8bd","Type":"ContainerDied","Data":"b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d"} Sep 15 07:26:33.133703 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:33.133363 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697b678444-96gjn" event={"ID":"16fb10e4-0872-4f43-adfe-d113e696b8bd","Type":"ContainerDied","Data":"b69302ab1ec3b416cea0ac138167870dda5184126378b3b419703a8e86caf8e2"} Sep 15 07:26:33.133703 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:33.133397 2552 scope.go:117] "RemoveContainer" containerID="b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d" Sep 15 07:26:33.141423 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:33.141404 2552 scope.go:117] "RemoveContainer" containerID="b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d" Sep 15 07:26:33.141668 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:26:33.141648 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d\": container with ID starting with b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d not found: ID does not exist" containerID="b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d" Sep 15 07:26:33.141754 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:33.141675 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d"} err="failed to get container status \"b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d\": rpc error: code = NotFound desc = could not find container \"b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d\": container with ID starting with b2d234fc16dfe176aacc04a3b3505b815f15eac729fc3efafc0e823410d3791d not found: ID does not exist" Sep 15 07:26:33.153783 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:33.153753 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-697b678444-96gjn"] Sep 15 07:26:33.157486 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:33.157464 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-697b678444-96gjn"] Sep 15 07:26:33.473659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:33.473585 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16fb10e4-0872-4f43-adfe-d113e696b8bd" path="/var/lib/kubelet/pods/16fb10e4-0872-4f43-adfe-d113e696b8bd/volumes" Sep 15 07:26:37.084554 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:37.084519 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:26:37.180297 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:37.180262 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:26:59.087621 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:26:59.087582 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:04.251567 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:04.251534 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:05.275427 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:05.275379 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:07.871516 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:07.871484 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:07.974561 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:07.974530 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:09.468117 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:09.468085 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:09.579064 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:09.579029 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:10.115049 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.115008 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:10.214642 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.214608 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:10.271580 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.271548 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-58mjn"] Sep 15 07:27:10.271847 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.271835 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16fb10e4-0872-4f43-adfe-d113e696b8bd" containerName="console" Sep 15 07:27:10.271922 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.271849 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="16fb10e4-0872-4f43-adfe-d113e696b8bd" containerName="console" Sep 15 07:27:10.271922 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.271913 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="16fb10e4-0872-4f43-adfe-d113e696b8bd" containerName="console" Sep 15 07:27:10.273721 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.273705 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" Sep 15 07:27:10.279624 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.279597 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 15 07:27:10.279821 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.279619 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 15 07:27:10.280959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.280739 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-jv2qz\"" Sep 15 07:27:10.290565 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.290544 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-58mjn"] Sep 15 07:27:10.367172 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.367075 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwv6x\" (UniqueName: \"kubernetes.io/projected/60e01985-c901-4e0b-85ca-a0152460da88-kube-api-access-fwv6x\") pod \"cert-manager-webhook-6486ffd997-58mjn\" (UID: \"60e01985-c901-4e0b-85ca-a0152460da88\") " pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" Sep 15 07:27:10.367336 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.367206 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60e01985-c901-4e0b-85ca-a0152460da88-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-58mjn\" (UID: \"60e01985-c901-4e0b-85ca-a0152460da88\") " pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" Sep 15 07:27:10.468208 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.468173 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60e01985-c901-4e0b-85ca-a0152460da88-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-58mjn\" (UID: \"60e01985-c901-4e0b-85ca-a0152460da88\") " pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" Sep 15 07:27:10.468573 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.468219 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwv6x\" (UniqueName: \"kubernetes.io/projected/60e01985-c901-4e0b-85ca-a0152460da88-kube-api-access-fwv6x\") pod \"cert-manager-webhook-6486ffd997-58mjn\" (UID: \"60e01985-c901-4e0b-85ca-a0152460da88\") " pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" Sep 15 07:27:10.477725 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.477691 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60e01985-c901-4e0b-85ca-a0152460da88-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-58mjn\" (UID: \"60e01985-c901-4e0b-85ca-a0152460da88\") " pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" Sep 15 07:27:10.485314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.485293 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwv6x\" (UniqueName: \"kubernetes.io/projected/60e01985-c901-4e0b-85ca-a0152460da88-kube-api-access-fwv6x\") pod \"cert-manager-webhook-6486ffd997-58mjn\" (UID: \"60e01985-c901-4e0b-85ca-a0152460da88\") " pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" Sep 15 07:27:10.588469 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.588433 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" Sep 15 07:27:10.859219 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:10.859041 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-58mjn"] Sep 15 07:27:10.861312 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:27:10.861284 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60e01985_c901_4e0b_85ca_a0152460da88.slice/crio-5bc5ced603555dfd3b877defe5995e2ef20f8538881af6d9bd8dff1e004c6723 WatchSource:0}: Error finding container 5bc5ced603555dfd3b877defe5995e2ef20f8538881af6d9bd8dff1e004c6723: Status 404 returned error can't find the container with id 5bc5ced603555dfd3b877defe5995e2ef20f8538881af6d9bd8dff1e004c6723 Sep 15 07:27:11.292154 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:11.292123 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" event={"ID":"60e01985-c901-4e0b-85ca-a0152460da88","Type":"ContainerStarted","Data":"5bc5ced603555dfd3b877defe5995e2ef20f8538881af6d9bd8dff1e004c6723"} Sep 15 07:27:12.315676 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:12.315583 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:12.683709 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:12.683617 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:14.307589 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:14.307560 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" event={"ID":"60e01985-c901-4e0b-85ca-a0152460da88","Type":"ContainerStarted","Data":"ff0e13fe3e65b2260766b629467415dc91da28caf6183be8c904005708c7bb85"} Sep 15 07:27:14.307978 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:14.307611 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" Sep 15 07:27:14.326075 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:14.326028 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" podStartSLOduration=1.412862127 podStartE2EDuration="4.326011346s" totalImagesPullingTime="2.913149219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:27:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:27:14.325357953 +0000 UTC m=+515.306520121" watchObservedRunningTime="2026-09-15 07:27:14.326011346 +0000 UTC m=+515.307173512" Sep 15 07:27:14.657465 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:14.657376 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:14.713699 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:14.713670 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:18.394700 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:18.394667 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sths5"] Sep 15 07:27:18.396689 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:18.396672 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sths5" Sep 15 07:27:18.399033 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:18.399015 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-b66nv\"" Sep 15 07:27:18.405645 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:18.405620 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sths5"] Sep 15 07:27:18.434769 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:18.434742 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzfpj\" (UniqueName: \"kubernetes.io/projected/1594ce66-4886-43cd-8d23-7b09b16d6901-kube-api-access-mzfpj\") pod \"cert-manager-cainjector-559cdd8c87-sths5\" (UID: \"1594ce66-4886-43cd-8d23-7b09b16d6901\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sths5" Sep 15 07:27:18.434883 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:18.434795 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1594ce66-4886-43cd-8d23-7b09b16d6901-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sths5\" (UID: \"1594ce66-4886-43cd-8d23-7b09b16d6901\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sths5" Sep 15 07:27:18.535566 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:18.535538 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1594ce66-4886-43cd-8d23-7b09b16d6901-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sths5\" (UID: \"1594ce66-4886-43cd-8d23-7b09b16d6901\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sths5" Sep 15 07:27:18.535683 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:18.535610 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzfpj\" (UniqueName: \"kubernetes.io/projected/1594ce66-4886-43cd-8d23-7b09b16d6901-kube-api-access-mzfpj\") pod \"cert-manager-cainjector-559cdd8c87-sths5\" (UID: \"1594ce66-4886-43cd-8d23-7b09b16d6901\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sths5" Sep 15 07:27:18.543737 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:18.543709 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1594ce66-4886-43cd-8d23-7b09b16d6901-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sths5\" (UID: \"1594ce66-4886-43cd-8d23-7b09b16d6901\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sths5" Sep 15 07:27:18.543941 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:18.543918 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzfpj\" (UniqueName: \"kubernetes.io/projected/1594ce66-4886-43cd-8d23-7b09b16d6901-kube-api-access-mzfpj\") pod \"cert-manager-cainjector-559cdd8c87-sths5\" (UID: \"1594ce66-4886-43cd-8d23-7b09b16d6901\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sths5" Sep 15 07:27:18.705280 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:18.705182 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sths5" Sep 15 07:27:18.975599 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:18.975421 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sths5"] Sep 15 07:27:18.978090 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:27:18.978063 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1594ce66_4886_43cd_8d23_7b09b16d6901.slice/crio-1aeacba0c8a44fb993eeb806716a0c0dee047bee561504b3ea274cb80e30df49 WatchSource:0}: Error finding container 1aeacba0c8a44fb993eeb806716a0c0dee047bee561504b3ea274cb80e30df49: Status 404 returned error can't find the container with id 1aeacba0c8a44fb993eeb806716a0c0dee047bee561504b3ea274cb80e30df49 Sep 15 07:27:19.332791 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:19.332696 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sths5" event={"ID":"1594ce66-4886-43cd-8d23-7b09b16d6901","Type":"ContainerStarted","Data":"b831e41441f31927ba95fdef4cbc25b25a2de89bc886e4dbdc5b84b4b1d98cb9"} Sep 15 07:27:19.332791 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:19.332753 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sths5" event={"ID":"1594ce66-4886-43cd-8d23-7b09b16d6901","Type":"ContainerStarted","Data":"1aeacba0c8a44fb993eeb806716a0c0dee047bee561504b3ea274cb80e30df49"} Sep 15 07:27:19.350024 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:19.349972 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sths5" podStartSLOduration=1.349953656 podStartE2EDuration="1.349953656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:27:19.347933126 +0000 UTC m=+520.329095298" watchObservedRunningTime="2026-09-15 07:27:19.349953656 +0000 UTC m=+520.331115825" Sep 15 07:27:19.743170 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:19.743136 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:19.801274 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:19.801233 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:20.314444 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:20.314415 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-58mjn" Sep 15 07:27:20.315568 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:20.315520 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:20.870046 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:20.870015 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:20.963233 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:20.963202 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:26.792394 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:26.792358 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:26.892523 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:26.892489 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:28.895344 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:28.895301 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:45.303879 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.303842 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:45.386149 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.386119 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r"] Sep 15 07:27:45.392053 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.392034 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.396404 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.396381 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 15 07:27:45.396521 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.396403 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 15 07:27:45.396521 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.396419 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 15 07:27:45.396604 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.396537 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-skwqt\"" Sep 15 07:27:45.397113 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.397093 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 15 07:27:45.406047 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.406029 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 15 07:27:45.408744 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.408719 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r"] Sep 15 07:27:45.415050 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.415024 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:45.436848 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.436819 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/812aaa8b-f2d2-4363-bf4d-f9ea0254519d-manager-config\") pod \"lws-controller-manager-7b9fc96c88-rrr4r\" (UID: \"812aaa8b-f2d2-4363-bf4d-f9ea0254519d\") " pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.436979 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.436885 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/812aaa8b-f2d2-4363-bf4d-f9ea0254519d-metrics-cert\") pod \"lws-controller-manager-7b9fc96c88-rrr4r\" (UID: \"812aaa8b-f2d2-4363-bf4d-f9ea0254519d\") " pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.436979 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.436939 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/812aaa8b-f2d2-4363-bf4d-f9ea0254519d-cert\") pod \"lws-controller-manager-7b9fc96c88-rrr4r\" (UID: \"812aaa8b-f2d2-4363-bf4d-f9ea0254519d\") " pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.437068 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.436996 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vfg5\" (UniqueName: \"kubernetes.io/projected/812aaa8b-f2d2-4363-bf4d-f9ea0254519d-kube-api-access-5vfg5\") pod \"lws-controller-manager-7b9fc96c88-rrr4r\" (UID: \"812aaa8b-f2d2-4363-bf4d-f9ea0254519d\") " pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.537566 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.537541 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/812aaa8b-f2d2-4363-bf4d-f9ea0254519d-manager-config\") pod \"lws-controller-manager-7b9fc96c88-rrr4r\" (UID: \"812aaa8b-f2d2-4363-bf4d-f9ea0254519d\") " pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.537728 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.537576 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/812aaa8b-f2d2-4363-bf4d-f9ea0254519d-metrics-cert\") pod \"lws-controller-manager-7b9fc96c88-rrr4r\" (UID: \"812aaa8b-f2d2-4363-bf4d-f9ea0254519d\") " pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.537728 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.537610 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/812aaa8b-f2d2-4363-bf4d-f9ea0254519d-cert\") pod \"lws-controller-manager-7b9fc96c88-rrr4r\" (UID: \"812aaa8b-f2d2-4363-bf4d-f9ea0254519d\") " pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.537728 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.537641 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vfg5\" (UniqueName: \"kubernetes.io/projected/812aaa8b-f2d2-4363-bf4d-f9ea0254519d-kube-api-access-5vfg5\") pod \"lws-controller-manager-7b9fc96c88-rrr4r\" (UID: \"812aaa8b-f2d2-4363-bf4d-f9ea0254519d\") " pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.538199 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.538178 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/812aaa8b-f2d2-4363-bf4d-f9ea0254519d-manager-config\") pod \"lws-controller-manager-7b9fc96c88-rrr4r\" (UID: \"812aaa8b-f2d2-4363-bf4d-f9ea0254519d\") " pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.540116 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.540094 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/812aaa8b-f2d2-4363-bf4d-f9ea0254519d-cert\") pod \"lws-controller-manager-7b9fc96c88-rrr4r\" (UID: \"812aaa8b-f2d2-4363-bf4d-f9ea0254519d\") " pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.540200 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.540117 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/812aaa8b-f2d2-4363-bf4d-f9ea0254519d-metrics-cert\") pod \"lws-controller-manager-7b9fc96c88-rrr4r\" (UID: \"812aaa8b-f2d2-4363-bf4d-f9ea0254519d\") " pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.562956 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.562902 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vfg5\" (UniqueName: \"kubernetes.io/projected/812aaa8b-f2d2-4363-bf4d-f9ea0254519d-kube-api-access-5vfg5\") pod \"lws-controller-manager-7b9fc96c88-rrr4r\" (UID: \"812aaa8b-f2d2-4363-bf4d-f9ea0254519d\") " pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.701120 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.701078 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:45.891339 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.891256 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:45.979762 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:45.979652 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r"] Sep 15 07:27:45.982242 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:27:45.982220 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod812aaa8b_f2d2_4363_bf4d_f9ea0254519d.slice/crio-e78d8bd143c90369467bc22ce779d67dd3faa9c55c44c0803932bdc398e2c10c WatchSource:0}: Error finding container e78d8bd143c90369467bc22ce779d67dd3faa9c55c44c0803932bdc398e2c10c: Status 404 returned error can't find the container with id e78d8bd143c90369467bc22ce779d67dd3faa9c55c44c0803932bdc398e2c10c Sep 15 07:27:46.361151 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.361119 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs"] Sep 15 07:27:46.368215 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.368183 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.372073 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.372046 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qd2gg\"" Sep 15 07:27:46.380289 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.380266 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 15 07:27:46.380428 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.380334 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 15 07:27:46.380494 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.380426 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 15 07:27:46.380547 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.380271 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 15 07:27:46.383182 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.383154 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs"] Sep 15 07:27:46.434526 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.434496 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" event={"ID":"812aaa8b-f2d2-4363-bf4d-f9ea0254519d","Type":"ContainerStarted","Data":"e78d8bd143c90369467bc22ce779d67dd3faa9c55c44c0803932bdc398e2c10c"} Sep 15 07:27:46.444820 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.444798 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4e90840b-cab1-407b-8fff-e1feac2c3a58-apiservice-cert\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.444931 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.444840 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pkww\" (UniqueName: \"kubernetes.io/projected/4e90840b-cab1-407b-8fff-e1feac2c3a58-kube-api-access-5pkww\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.444931 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.444892 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e90840b-cab1-407b-8fff-e1feac2c3a58-webhook-cert\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.445020 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.444946 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4e90840b-cab1-407b-8fff-e1feac2c3a58-manifests\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.445020 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.445007 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e90840b-cab1-407b-8fff-e1feac2c3a58-tmp\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.546281 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.546248 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e90840b-cab1-407b-8fff-e1feac2c3a58-tmp\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.546418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.546295 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4e90840b-cab1-407b-8fff-e1feac2c3a58-apiservice-cert\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.546418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.546332 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pkww\" (UniqueName: \"kubernetes.io/projected/4e90840b-cab1-407b-8fff-e1feac2c3a58-kube-api-access-5pkww\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.546418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.546368 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e90840b-cab1-407b-8fff-e1feac2c3a58-webhook-cert\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.546571 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.546469 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4e90840b-cab1-407b-8fff-e1feac2c3a58-manifests\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.546848 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.546819 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4e90840b-cab1-407b-8fff-e1feac2c3a58-manifests\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.548533 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.548505 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e90840b-cab1-407b-8fff-e1feac2c3a58-tmp\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.548787 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.548767 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4e90840b-cab1-407b-8fff-e1feac2c3a58-apiservice-cert\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.548862 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.548767 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e90840b-cab1-407b-8fff-e1feac2c3a58-webhook-cert\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.556853 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.556833 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pkww\" (UniqueName: \"kubernetes.io/projected/4e90840b-cab1-407b-8fff-e1feac2c3a58-kube-api-access-5pkww\") pod \"opendatahub-operator-controller-manager-565d545f7b-lpnqs\" (UID: \"4e90840b-cab1-407b-8fff-e1feac2c3a58\") " pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.689420 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.689352 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:46.968831 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:46.968787 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs"] Sep 15 07:27:46.971713 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:27:46.971679 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e90840b_cab1_407b_8fff_e1feac2c3a58.slice/crio-6310ee479f8a383e8a8c084c0518a58222dd6eac993cbb98b416f117e3ca589c WatchSource:0}: Error finding container 6310ee479f8a383e8a8c084c0518a58222dd6eac993cbb98b416f117e3ca589c: Status 404 returned error can't find the container with id 6310ee479f8a383e8a8c084c0518a58222dd6eac993cbb98b416f117e3ca589c Sep 15 07:27:47.441147 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:47.441110 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" event={"ID":"4e90840b-cab1-407b-8fff-e1feac2c3a58","Type":"ContainerStarted","Data":"6310ee479f8a383e8a8c084c0518a58222dd6eac993cbb98b416f117e3ca589c"} Sep 15 07:27:48.009654 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:48.009619 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:27:51.462997 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:51.462964 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" event={"ID":"4e90840b-cab1-407b-8fff-e1feac2c3a58","Type":"ContainerStarted","Data":"3dab599343b413832fe4ae2ecea670a4a8efbc388677cbf828655d21d5e3ffed"} Sep 15 07:27:51.464520 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:51.464498 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" event={"ID":"812aaa8b-f2d2-4363-bf4d-f9ea0254519d","Type":"ContainerStarted","Data":"ce0e68416eba4237daea51af05793ed10d7e997011db4408bd2159cbcfc4939c"} Sep 15 07:27:51.464618 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:51.464554 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:27:51.503474 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:51.503432 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" podStartSLOduration=2.099366616 podStartE2EDuration="6.503417365s" totalImagesPullingTime="4.404050749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:27:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:27:51.501672063 +0000 UTC m=+552.482834230" watchObservedRunningTime="2026-09-15 07:27:51.503417365 +0000 UTC m=+552.484579532" Sep 15 07:27:52.471085 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:52.471046 2552 generic.go:356] "Generic (PLEG): container finished" podID="4e90840b-cab1-407b-8fff-e1feac2c3a58" containerID="3dab599343b413832fe4ae2ecea670a4a8efbc388677cbf828655d21d5e3ffed" exitCode=0 Sep 15 07:27:52.471473 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:52.471129 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" event={"ID":"4e90840b-cab1-407b-8fff-e1feac2c3a58","Type":"ContainerDied","Data":"3dab599343b413832fe4ae2ecea670a4a8efbc388677cbf828655d21d5e3ffed"} Sep 15 07:27:53.477975 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:53.477937 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" event={"ID":"4e90840b-cab1-407b-8fff-e1feac2c3a58","Type":"ContainerStarted","Data":"e0b7603db0d18fd3272a41a2202fded80052792be58b09d4dad37378259e24e8"} Sep 15 07:27:53.478496 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:53.478070 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:27:53.500727 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:27:53.500669 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" podStartSLOduration=2.657509491 podStartE2EDuration="7.500647985s" totalImagesPullingTime="3.85100757s" totalInitContainerRuntime="992.130924ms" isStatefulPod=false podCreationTimestamp="2026-09-15 07:27:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:27:53.498966739 +0000 UTC m=+554.480128906" watchObservedRunningTime="2026-09-15 07:27:53.500647985 +0000 UTC m=+554.481810152" Sep 15 07:28:01.289315 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:01.289277 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:01.381413 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:01.381368 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:02.474282 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:02.474252 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b9fc96c88-rrr4r" Sep 15 07:28:04.485187 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:04.485155 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-565d545f7b-lpnqs" Sep 15 07:28:07.919775 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:07.919742 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:08.038131 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:08.038089 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:08.248823 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:08.248789 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:08.335092 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:08.335053 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:09.459400 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:09.459360 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:09.558118 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:09.558080 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:10.506328 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:10.506296 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:13.790849 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:13.790813 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:13.892361 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:13.892288 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:14.445559 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.445525 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt"] Sep 15 07:28:14.449411 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.449390 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.451712 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.451688 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 15 07:28:14.451822 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.451800 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-wllnq\"" Sep 15 07:28:14.463237 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.463210 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt"] Sep 15 07:28:14.486105 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.486069 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:14.596894 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.596840 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.597067 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.596902 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.597067 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.596930 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.597067 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.596957 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.597067 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.596979 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.597221 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.597097 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.597221 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.597124 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.597221 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.597140 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.597221 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.597155 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqrk8\" (UniqueName: \"kubernetes.io/projected/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-kube-api-access-mqrk8\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.674220 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.674186 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:14.698374 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.698309 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.698374 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.698340 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.698374 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.698364 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.698601 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.698393 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.698601 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.698416 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.698601 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.698481 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.698601 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.698504 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.698601 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.698527 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.698601 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.698550 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqrk8\" (UniqueName: \"kubernetes.io/projected/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-kube-api-access-mqrk8\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.698934 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.698748 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.698934 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.698832 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.699117 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.699094 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.699212 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.699155 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.699293 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.699271 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.701087 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.701060 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.701286 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.701264 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.706688 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.706668 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.706952 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.706933 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqrk8\" (UniqueName: \"kubernetes.io/projected/024f5b7f-74ab-4aaf-9dc9-d9bf9e089707-kube-api-access-mqrk8\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt\" (UID: \"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.760851 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.760820 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:14.774851 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:14.774818 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:15.037332 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:15.037308 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt"] Sep 15 07:28:15.038978 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:28:15.038946 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod024f5b7f_74ab_4aaf_9dc9_d9bf9e089707.slice/crio-cffe51c3fff0ad2b1b8f8a39f63d250b3a03fd99aeaeb943e91efbdc5bae4f15 WatchSource:0}: Error finding container cffe51c3fff0ad2b1b8f8a39f63d250b3a03fd99aeaeb943e91efbdc5bae4f15: Status 404 returned error can't find the container with id cffe51c3fff0ad2b1b8f8a39f63d250b3a03fd99aeaeb943e91efbdc5bae4f15 Sep 15 07:28:15.572528 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:15.572496 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" event={"ID":"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707","Type":"ContainerStarted","Data":"cffe51c3fff0ad2b1b8f8a39f63d250b3a03fd99aeaeb943e91efbdc5bae4f15"} Sep 15 07:28:17.528821 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:17.528769 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:28:17.529118 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:17.528858 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:28:17.529118 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:17.528909 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:28:18.593055 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:18.593018 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" event={"ID":"024f5b7f-74ab-4aaf-9dc9-d9bf9e089707","Type":"ContainerStarted","Data":"c86e49d24eebaed63e1b613738ba42a145b05c218e57be859ee7cfd7bd025e89"} Sep 15 07:28:18.621691 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:18.621636 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" podStartSLOduration=2.133871739 podStartE2EDuration="4.621619139s" totalImagesPullingTime="2.4877474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:28:18.621302558 +0000 UTC m=+579.602464726" watchObservedRunningTime="2026-09-15 07:28:18.621619139 +0000 UTC m=+579.602781307" Sep 15 07:28:18.761331 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:18.761297 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:18.766303 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:18.766278 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:18.873456 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:18.873380 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:19.161534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.161451 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jzhtd"] Sep 15 07:28:19.169898 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.169853 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" Sep 15 07:28:19.172174 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.172148 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 15 07:28:19.172317 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.172194 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-r6mgd\"" Sep 15 07:28:19.175664 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.175636 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jzhtd"] Sep 15 07:28:19.238342 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.238303 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlxfg\" (UniqueName: \"kubernetes.io/projected/5b648781-f1de-428f-8606-9861f52bb7db-kube-api-access-jlxfg\") pod \"model-serving-api-54d89bd5fc-jzhtd\" (UID: \"5b648781-f1de-428f-8606-9861f52bb7db\") " pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" Sep 15 07:28:19.238524 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.238358 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b648781-f1de-428f-8606-9861f52bb7db-tls-certs\") pod \"model-serving-api-54d89bd5fc-jzhtd\" (UID: \"5b648781-f1de-428f-8606-9861f52bb7db\") " pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" Sep 15 07:28:19.339485 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.339446 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlxfg\" (UniqueName: \"kubernetes.io/projected/5b648781-f1de-428f-8606-9861f52bb7db-kube-api-access-jlxfg\") pod \"model-serving-api-54d89bd5fc-jzhtd\" (UID: \"5b648781-f1de-428f-8606-9861f52bb7db\") " pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" Sep 15 07:28:19.339662 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.339502 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b648781-f1de-428f-8606-9861f52bb7db-tls-certs\") pod \"model-serving-api-54d89bd5fc-jzhtd\" (UID: \"5b648781-f1de-428f-8606-9861f52bb7db\") " pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" Sep 15 07:28:19.341879 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.341844 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b648781-f1de-428f-8606-9861f52bb7db-tls-certs\") pod \"model-serving-api-54d89bd5fc-jzhtd\" (UID: \"5b648781-f1de-428f-8606-9861f52bb7db\") " pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" Sep 15 07:28:19.347710 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.347676 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlxfg\" (UniqueName: \"kubernetes.io/projected/5b648781-f1de-428f-8606-9861f52bb7db-kube-api-access-jlxfg\") pod \"model-serving-api-54d89bd5fc-jzhtd\" (UID: \"5b648781-f1de-428f-8606-9861f52bb7db\") " pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" Sep 15 07:28:19.482655 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.482618 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" Sep 15 07:28:19.598378 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.598348 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:19.599538 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.599515 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt" Sep 15 07:28:19.758401 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:19.758377 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jzhtd"] Sep 15 07:28:19.760358 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:28:19.760334 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b648781_f1de_428f_8606_9861f52bb7db.slice/crio-2de1e24ea9e48d472d5129d08ed0fcb04839817e92291224de34ab4abf0568ec WatchSource:0}: Error finding container 2de1e24ea9e48d472d5129d08ed0fcb04839817e92291224de34ab4abf0568ec: Status 404 returned error can't find the container with id 2de1e24ea9e48d472d5129d08ed0fcb04839817e92291224de34ab4abf0568ec Sep 15 07:28:20.606656 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:20.606571 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" event={"ID":"5b648781-f1de-428f-8606-9861f52bb7db","Type":"ContainerStarted","Data":"2de1e24ea9e48d472d5129d08ed0fcb04839817e92291224de34ab4abf0568ec"} Sep 15 07:28:21.795455 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:21.795413 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:21.851597 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:21.851555 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:22.619731 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:22.619650 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" event={"ID":"5b648781-f1de-428f-8606-9861f52bb7db","Type":"ContainerStarted","Data":"9753088a726b36e6f25f32ba62fc3585dae17833fbc28e77c342d9497513bfc5"} Sep 15 07:28:22.619906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:22.619770 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" Sep 15 07:28:22.635591 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:22.635547 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" podStartSLOduration=1.12567935 podStartE2EDuration="3.635530512s" totalImagesPullingTime="2.509851162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:28:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:28:22.634951724 +0000 UTC m=+583.616113891" watchObservedRunningTime="2026-09-15 07:28:22.635530512 +0000 UTC m=+583.616692700" Sep 15 07:28:25.339977 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.339944 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:25.401399 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.401367 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:25.973120 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.973083 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct"] Sep 15 07:28:25.979561 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.979532 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:25.982493 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.982471 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 15 07:28:25.982759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.982744 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 15 07:28:25.982993 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.982978 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 15 07:28:25.983303 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.983288 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fdlzs\"" Sep 15 07:28:25.992885 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.992839 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct"] Sep 15 07:28:25.997397 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.997376 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6091e72d-1db6-4b32-9420-f1cd119c990b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:25.997526 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.997404 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6091e72d-1db6-4b32-9420-f1cd119c990b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:25.997526 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.997423 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6091e72d-1db6-4b32-9420-f1cd119c990b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:25.997526 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.997440 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6091e72d-1db6-4b32-9420-f1cd119c990b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:25.997701 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.997535 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6091e72d-1db6-4b32-9420-f1cd119c990b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:25.997701 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.997554 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpgxm\" (UniqueName: \"kubernetes.io/projected/6091e72d-1db6-4b32-9420-f1cd119c990b-kube-api-access-gpgxm\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:25.997701 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:25.997584 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6091e72d-1db6-4b32-9420-f1cd119c990b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.098445 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.098413 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6091e72d-1db6-4b32-9420-f1cd119c990b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.098606 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.098471 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6091e72d-1db6-4b32-9420-f1cd119c990b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.098606 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.098490 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6091e72d-1db6-4b32-9420-f1cd119c990b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.098606 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.098505 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6091e72d-1db6-4b32-9420-f1cd119c990b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.098606 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.098522 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6091e72d-1db6-4b32-9420-f1cd119c990b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.098606 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.098578 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6091e72d-1db6-4b32-9420-f1cd119c990b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.098854 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.098776 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpgxm\" (UniqueName: \"kubernetes.io/projected/6091e72d-1db6-4b32-9420-f1cd119c990b-kube-api-access-gpgxm\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.099230 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.099203 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6091e72d-1db6-4b32-9420-f1cd119c990b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.100817 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.100797 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6091e72d-1db6-4b32-9420-f1cd119c990b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.101120 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.101097 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6091e72d-1db6-4b32-9420-f1cd119c990b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.101400 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.101381 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6091e72d-1db6-4b32-9420-f1cd119c990b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.101441 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.101389 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6091e72d-1db6-4b32-9420-f1cd119c990b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.109261 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.109236 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpgxm\" (UniqueName: \"kubernetes.io/projected/6091e72d-1db6-4b32-9420-f1cd119c990b-kube-api-access-gpgxm\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.110037 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.110013 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6091e72d-1db6-4b32-9420-f1cd119c990b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-42jct\" (UID: \"6091e72d-1db6-4b32-9420-f1cd119c990b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.289362 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.289338 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:26.565751 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.565720 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct"] Sep 15 07:28:26.567000 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:28:26.566968 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6091e72d_1db6_4b32_9420_f1cd119c990b.slice/crio-8f4f4ecd1d42a0aa8cb8326fc78fbf87f332977f5a1f72273d519b8de1020a27 WatchSource:0}: Error finding container 8f4f4ecd1d42a0aa8cb8326fc78fbf87f332977f5a1f72273d519b8de1020a27: Status 404 returned error can't find the container with id 8f4f4ecd1d42a0aa8cb8326fc78fbf87f332977f5a1f72273d519b8de1020a27 Sep 15 07:28:26.640171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:26.640137 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" event={"ID":"6091e72d-1db6-4b32-9420-f1cd119c990b","Type":"ContainerStarted","Data":"8f4f4ecd1d42a0aa8cb8326fc78fbf87f332977f5a1f72273d519b8de1020a27"} Sep 15 07:28:28.632324 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:28.632289 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:28.720609 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:28.720575 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:28.980442 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:28.980409 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:29.116263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:29.116214 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:29.181553 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:29.181515 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:31.956153 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:31.956113 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:28:31.956523 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:31.956193 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:28:32.674768 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:32.674734 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" event={"ID":"6091e72d-1db6-4b32-9420-f1cd119c990b","Type":"ContainerStarted","Data":"cbd7e43e040ceb61c6c9e5b5807ee4f7941bfed95fdfeda3b46b613e1039787b"} Sep 15 07:28:32.675070 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:32.674856 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:32.701390 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:32.698339 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" podStartSLOduration=2.311725679 podStartE2EDuration="7.69831736s" totalImagesPullingTime="5.386591681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:28:32.695191081 +0000 UTC m=+593.676353249" watchObservedRunningTime="2026-09-15 07:28:32.69831736 +0000 UTC m=+593.679479528" Sep 15 07:28:33.630597 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:33.630564 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-jzhtd" Sep 15 07:28:33.680756 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:33.680725 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-42jct" Sep 15 07:28:35.137086 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:35.137054 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:35.190734 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:35.190705 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:38.639247 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:38.639210 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:38.692491 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:38.692451 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:40.983195 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:40.983169 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:28:40.991040 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:40.984166 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:28:40.991040 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:40.989789 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:28:40.991040 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:40.990722 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:28:45.578153 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:45.578118 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b965b86cb-7ntcg"] Sep 15 07:28:48.697805 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:48.697773 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:28:48.748836 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:28:48.748806 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:10.600643 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.600604 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b965b86cb-7ntcg" podUID="fc7ead54-7930-4550-9baa-990d86fdf2ce" containerName="console" containerID="cri-o://251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201" gracePeriod=15 Sep 15 07:29:10.839990 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.839964 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:29:10.841171 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.841145 2552 generic.go:356] "Generic (PLEG): container finished" podID="fc7ead54-7930-4550-9baa-990d86fdf2ce" containerID="251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201" exitCode=0 Sep 15 07:29:10.841263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.841213 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b965b86cb-7ntcg" event={"ID":"fc7ead54-7930-4550-9baa-990d86fdf2ce","Type":"ContainerDied","Data":"251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201"} Sep 15 07:29:10.841263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.841249 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b965b86cb-7ntcg" event={"ID":"fc7ead54-7930-4550-9baa-990d86fdf2ce","Type":"ContainerDied","Data":"645faa2110fde24773afad02cc2a740ff6f5f774e2d68ccd9839000f94a9024f"} Sep 15 07:29:10.841355 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.841268 2552 scope.go:117] "RemoveContainer" containerID="251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201" Sep 15 07:29:10.849355 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.849337 2552 scope.go:117] "RemoveContainer" containerID="251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201" Sep 15 07:29:10.849604 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:29:10.849587 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201\": container with ID starting with 251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201 not found: ID does not exist" containerID="251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201" Sep 15 07:29:10.849649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.849613 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201"} err="failed to get container status \"251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201\": rpc error: code = NotFound desc = could not find container \"251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201\": container with ID starting with 251d2e8e86f5ab061708930c22cb29fca9cdc2572beb787d04a9bf3ef51e1201 not found: ID does not exist" Sep 15 07:29:10.891342 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.891249 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-service-ca\") pod \"fc7ead54-7930-4550-9baa-990d86fdf2ce\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " Sep 15 07:29:10.891342 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.891291 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxpzp\" (UniqueName: \"kubernetes.io/projected/fc7ead54-7930-4550-9baa-990d86fdf2ce-kube-api-access-wxpzp\") pod \"fc7ead54-7930-4550-9baa-990d86fdf2ce\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " Sep 15 07:29:10.891342 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.891331 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-oauth-serving-cert\") pod \"fc7ead54-7930-4550-9baa-990d86fdf2ce\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " Sep 15 07:29:10.891620 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.891380 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-oauth-config\") pod \"fc7ead54-7930-4550-9baa-990d86fdf2ce\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " Sep 15 07:29:10.891620 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.891433 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-trusted-ca-bundle\") pod \"fc7ead54-7930-4550-9baa-990d86fdf2ce\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " Sep 15 07:29:10.891620 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.891456 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-config\") pod \"fc7ead54-7930-4550-9baa-990d86fdf2ce\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " Sep 15 07:29:10.891620 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.891479 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-serving-cert\") pod \"fc7ead54-7930-4550-9baa-990d86fdf2ce\" (UID: \"fc7ead54-7930-4550-9baa-990d86fdf2ce\") " Sep 15 07:29:10.891908 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.891790 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-service-ca" (OuterVolumeSpecName: "service-ca") pod "fc7ead54-7930-4550-9baa-990d86fdf2ce" (UID: "fc7ead54-7930-4550-9baa-990d86fdf2ce"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:29:10.891983 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.891932 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fc7ead54-7930-4550-9baa-990d86fdf2ce" (UID: "fc7ead54-7930-4550-9baa-990d86fdf2ce"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:29:10.891983 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.891948 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-config" (OuterVolumeSpecName: "console-config") pod "fc7ead54-7930-4550-9baa-990d86fdf2ce" (UID: "fc7ead54-7930-4550-9baa-990d86fdf2ce"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:29:10.892083 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.891953 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fc7ead54-7930-4550-9baa-990d86fdf2ce" (UID: "fc7ead54-7930-4550-9baa-990d86fdf2ce"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:29:10.893585 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.893556 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fc7ead54-7930-4550-9baa-990d86fdf2ce" (UID: "fc7ead54-7930-4550-9baa-990d86fdf2ce"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:29:10.893690 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.893618 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fc7ead54-7930-4550-9baa-990d86fdf2ce" (UID: "fc7ead54-7930-4550-9baa-990d86fdf2ce"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:29:10.893690 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.893628 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc7ead54-7930-4550-9baa-990d86fdf2ce-kube-api-access-wxpzp" (OuterVolumeSpecName: "kube-api-access-wxpzp") pod "fc7ead54-7930-4550-9baa-990d86fdf2ce" (UID: "fc7ead54-7930-4550-9baa-990d86fdf2ce"). InnerVolumeSpecName "kube-api-access-wxpzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:29:10.992150 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.992114 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-trusted-ca-bundle\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:29:10.992150 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.992146 2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:29:10.992368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.992159 2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-serving-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:29:10.992368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.992170 2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:29:10.992368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.992182 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxpzp\" (UniqueName: \"kubernetes.io/projected/fc7ead54-7930-4550-9baa-990d86fdf2ce-kube-api-access-wxpzp\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:29:10.992368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.992194 2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc7ead54-7930-4550-9baa-990d86fdf2ce-oauth-serving-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:29:10.992368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:10.992204 2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc7ead54-7930-4550-9baa-990d86fdf2ce-console-oauth-config\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:29:11.848144 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:11.848060 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b965b86cb-7ntcg" Sep 15 07:29:11.864312 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:11.864275 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b965b86cb-7ntcg"] Sep 15 07:29:11.867775 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:11.867741 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b965b86cb-7ntcg"] Sep 15 07:29:13.473303 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:13.473274 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc7ead54-7930-4550-9baa-990d86fdf2ce" path="/var/lib/kubelet/pods/fc7ead54-7930-4550-9baa-990d86fdf2ce/volumes" Sep 15 07:29:19.914614 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:19.914578 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:20.010216 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:20.010182 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:25.958824 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:25.958780 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:34.573378 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:34.573339 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:37.171917 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:37.171883 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:37.590589 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:37.590556 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:37.966316 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:37.966227 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:39.076231 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:39.076192 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:40.847442 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:40.847408 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:40.945522 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:40.945485 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:41.084701 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:41.084667 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:41.398106 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:41.398075 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:42.402112 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:42.402075 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:42.526105 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:42.526079 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:43.988137 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:43.988103 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:44.226172 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:44.226132 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:46.982290 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:46.982256 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:47.083093 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:47.083058 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:47.892349 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:47.892306 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:48.051852 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:48.051815 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:48.153178 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:48.153095 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:48.324290 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:48.324260 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:49.807467 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:49.807425 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:52.368540 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:52.368508 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:52.432685 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:52.432657 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:52.563020 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:52.562984 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:52.619430 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:52.619344 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:52.765313 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:52.765280 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:29:52.827238 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:29:52.827203 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:00.393456 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:00.393421 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:00.497893 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:00.497836 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:02.058331 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:02.058296 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:03.373787 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:03.373756 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:03.475436 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:03.475404 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:03.721060 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:03.720963 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:05.974483 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:05.974446 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:06.079074 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:06.079035 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:11.373936 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:11.373899 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:11.448739 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:11.448706 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:11.774902 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:11.774857 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:11.838852 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:11.838809 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:12.262732 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:12.262696 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:12.363499 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:12.363470 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:20.177634 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:20.177597 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:20.505833 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:20.505769 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:23.180157 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:23.180120 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:23.282185 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:23.282160 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:28.272882 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:28.272822 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:28.593528 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:28.593447 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:34.223571 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:34.223465 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:34.291125 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:34.291094 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:34.347094 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:34.347063 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:34.991076 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:34.991043 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:38.210670 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:38.210640 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:50.186829 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:50.186787 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:50.286511 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:50.286480 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:30:50.623022 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:30:50.622994 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:02.600881 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:02.600842 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:02.697919 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:02.697889 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:09.689552 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.689512 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:09.723527 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.723492 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d6956c67b-6dvc5"] Sep 15 07:31:09.723835 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.723822 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc7ead54-7930-4550-9baa-990d86fdf2ce" containerName="console" Sep 15 07:31:09.723835 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.723836 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc7ead54-7930-4550-9baa-990d86fdf2ce" containerName="console" Sep 15 07:31:09.723984 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.723934 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc7ead54-7930-4550-9baa-990d86fdf2ce" containerName="console" Sep 15 07:31:09.726658 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.726642 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:09.729388 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.729369 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 15 07:31:09.729388 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.729377 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 15 07:31:09.729785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.729769 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 15 07:31:09.729841 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.729769 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vw7xh\"" Sep 15 07:31:09.739641 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.739617 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6956c67b-6dvc5"] Sep 15 07:31:09.759922 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.759894 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:09.810795 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.810760 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-metrics-certs\") pod \"maas-controller-d6956c67b-6dvc5\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:09.810996 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.810818 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-webhook-cert\") pod \"maas-controller-d6956c67b-6dvc5\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:09.810996 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.810882 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncf82\" (UniqueName: \"kubernetes.io/projected/ee48f71a-3684-4375-9778-3034343c9f86-kube-api-access-ncf82\") pod \"maas-controller-d6956c67b-6dvc5\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:09.815437 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.815409 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:09.899878 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.899824 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:09.911970 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.911940 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-metrics-certs\") pod \"maas-controller-d6956c67b-6dvc5\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:09.912116 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.911989 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-webhook-cert\") pod \"maas-controller-d6956c67b-6dvc5\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:09.912116 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.912027 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncf82\" (UniqueName: \"kubernetes.io/projected/ee48f71a-3684-4375-9778-3034343c9f86-kube-api-access-ncf82\") pod \"maas-controller-d6956c67b-6dvc5\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:09.912213 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:31:09.912147 2552 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 15 07:31:09.912255 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:31:09.912219 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-webhook-cert podName:ee48f71a-3684-4375-9778-3034343c9f86 nodeName:}" failed. No retries permitted until 2026-09-15 07:31:10.412203222 +0000 UTC m=+751.393365383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-webhook-cert") pod "maas-controller-d6956c67b-6dvc5" (UID: "ee48f71a-3684-4375-9778-3034343c9f86") : secret "maas-controller-webhook-cert" not found Sep 15 07:31:09.914407 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.914387 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-metrics-certs\") pod \"maas-controller-d6956c67b-6dvc5\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:09.923015 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.922993 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncf82\" (UniqueName: \"kubernetes.io/projected/ee48f71a-3684-4375-9778-3034343c9f86-kube-api-access-ncf82\") pod \"maas-controller-d6956c67b-6dvc5\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:09.964590 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:09.964524 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:10.415503 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:10.415473 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-webhook-cert\") pod \"maas-controller-d6956c67b-6dvc5\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:10.417881 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:10.417848 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-webhook-cert\") pod \"maas-controller-d6956c67b-6dvc5\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:10.637793 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:10.637756 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:10.907970 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:10.907942 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d6956c67b-6dvc5"] Sep 15 07:31:10.910570 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:31:10.910542 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee48f71a_3684_4375_9778_3034343c9f86.slice/crio-033e24b5dfb6b137f51e68e6c3f24488987d3d6430d2e0e46f87f9b7b5a3e926 WatchSource:0}: Error finding container 033e24b5dfb6b137f51e68e6c3f24488987d3d6430d2e0e46f87f9b7b5a3e926: Status 404 returned error can't find the container with id 033e24b5dfb6b137f51e68e6c3f24488987d3d6430d2e0e46f87f9b7b5a3e926 Sep 15 07:31:10.911820 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:10.911798 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 07:31:11.384477 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:11.384444 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6956c67b-6dvc5" event={"ID":"ee48f71a-3684-4375-9778-3034343c9f86","Type":"ContainerStarted","Data":"033e24b5dfb6b137f51e68e6c3f24488987d3d6430d2e0e46f87f9b7b5a3e926"} Sep 15 07:31:13.410103 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:13.410068 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:13.515674 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:13.515639 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:14.403093 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:14.403062 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6956c67b-6dvc5" event={"ID":"ee48f71a-3684-4375-9778-3034343c9f86","Type":"ContainerStarted","Data":"521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8"} Sep 15 07:31:14.403255 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:14.403158 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:14.420459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:14.420410 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d6956c67b-6dvc5" podStartSLOduration=2.792214085 podStartE2EDuration="5.420394229s" totalImagesPullingTime="2.628180144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:31:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:31:14.418768121 +0000 UTC m=+755.399930353" watchObservedRunningTime="2026-09-15 07:31:14.420394229 +0000 UTC m=+755.401556396" Sep 15 07:31:24.410844 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.410813 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d6956c67b-6dvc5"] Sep 15 07:31:24.411270 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.411003 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-d6956c67b-6dvc5" podUID="ee48f71a-3684-4375-9778-3034343c9f86" containerName="manager" containerID="cri-o://521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8" gracePeriod=30 Sep 15 07:31:24.413031 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.413003 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:24.658691 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.658667 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:24.846037 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.846011 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncf82\" (UniqueName: \"kubernetes.io/projected/ee48f71a-3684-4375-9778-3034343c9f86-kube-api-access-ncf82\") pod \"ee48f71a-3684-4375-9778-3034343c9f86\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " Sep 15 07:31:24.846213 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.846131 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-metrics-certs\") pod \"ee48f71a-3684-4375-9778-3034343c9f86\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " Sep 15 07:31:24.846213 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.846148 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-webhook-cert\") pod \"ee48f71a-3684-4375-9778-3034343c9f86\" (UID: \"ee48f71a-3684-4375-9778-3034343c9f86\") " Sep 15 07:31:24.848225 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.848195 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ee48f71a-3684-4375-9778-3034343c9f86" (UID: "ee48f71a-3684-4375-9778-3034343c9f86"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:31:24.848340 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.848271 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ee48f71a-3684-4375-9778-3034343c9f86" (UID: "ee48f71a-3684-4375-9778-3034343c9f86"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:31:24.848340 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.848283 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee48f71a-3684-4375-9778-3034343c9f86-kube-api-access-ncf82" (OuterVolumeSpecName: "kube-api-access-ncf82") pod "ee48f71a-3684-4375-9778-3034343c9f86" (UID: "ee48f71a-3684-4375-9778-3034343c9f86"). InnerVolumeSpecName "kube-api-access-ncf82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:31:24.946908 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.946843 2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-metrics-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:31:24.946908 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.946905 2552 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee48f71a-3684-4375-9778-3034343c9f86-webhook-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:31:24.946908 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:24.946915 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncf82\" (UniqueName: \"kubernetes.io/projected/ee48f71a-3684-4375-9778-3034343c9f86-kube-api-access-ncf82\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:31:25.454850 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:25.454809 2552 generic.go:356] "Generic (PLEG): container finished" podID="ee48f71a-3684-4375-9778-3034343c9f86" containerID="521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8" exitCode=0 Sep 15 07:31:25.455321 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:25.454855 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d6956c67b-6dvc5" Sep 15 07:31:25.455321 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:25.454858 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6956c67b-6dvc5" event={"ID":"ee48f71a-3684-4375-9778-3034343c9f86","Type":"ContainerDied","Data":"521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8"} Sep 15 07:31:25.455321 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:25.454963 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d6956c67b-6dvc5" event={"ID":"ee48f71a-3684-4375-9778-3034343c9f86","Type":"ContainerDied","Data":"033e24b5dfb6b137f51e68e6c3f24488987d3d6430d2e0e46f87f9b7b5a3e926"} Sep 15 07:31:25.455321 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:25.454992 2552 scope.go:117] "RemoveContainer" containerID="521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8" Sep 15 07:31:25.464314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:25.464295 2552 scope.go:117] "RemoveContainer" containerID="521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8" Sep 15 07:31:25.464571 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:31:25.464552 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8\": container with ID starting with 521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8 not found: ID does not exist" containerID="521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8" Sep 15 07:31:25.464631 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:25.464582 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8"} err="failed to get container status \"521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8\": rpc error: code = NotFound desc = could not find container \"521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8\": container with ID starting with 521aa1b45134a62b29a54e449deb81d1e233603a3f3215b36cc87a88425ea6e8 not found: ID does not exist" Sep 15 07:31:25.475730 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:25.475707 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d6956c67b-6dvc5"] Sep 15 07:31:25.477699 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:25.477677 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d6956c67b-6dvc5"] Sep 15 07:31:27.473682 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:27.473643 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee48f71a-3684-4375-9778-3034343c9f86" path="/var/lib/kubelet/pods/ee48f71a-3684-4375-9778-3034343c9f86/volumes" Sep 15 07:31:41.432149 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:41.432114 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:44.810828 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:44.810797 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:44.930138 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:44.930097 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:46.302837 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:46.302791 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:46.397115 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:46.397076 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:46.539244 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:46.539210 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:46.639197 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:46.639122 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:46.889698 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:46.889613 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:46.991418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:46.991385 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:47.010807 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.010779 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-cgt48"] Sep 15 07:31:47.011126 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.011112 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee48f71a-3684-4375-9778-3034343c9f86" containerName="manager" Sep 15 07:31:47.011182 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.011128 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee48f71a-3684-4375-9778-3034343c9f86" containerName="manager" Sep 15 07:31:47.011225 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.011187 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee48f71a-3684-4375-9778-3034343c9f86" containerName="manager" Sep 15 07:31:47.019120 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.019101 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" Sep 15 07:31:47.041316 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.041296 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nk4q4\"" Sep 15 07:31:47.041445 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.041429 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-cgt48"] Sep 15 07:31:47.044883 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.044847 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 15 07:31:47.134110 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.134076 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pb29\" (UniqueName: \"kubernetes.io/projected/11660574-efac-4686-962b-934e3bd76287-kube-api-access-7pb29\") pod \"payload-processing-5b54b7fbb7-cgt48\" (UID: \"11660574-efac-4686-962b-934e3bd76287\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" Sep 15 07:31:47.134278 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.134119 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11660574-efac-4686-962b-934e3bd76287-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-cgt48\" (UID: \"11660574-efac-4686-962b-934e3bd76287\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" Sep 15 07:31:47.234813 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.234780 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pb29\" (UniqueName: \"kubernetes.io/projected/11660574-efac-4686-962b-934e3bd76287-kube-api-access-7pb29\") pod \"payload-processing-5b54b7fbb7-cgt48\" (UID: \"11660574-efac-4686-962b-934e3bd76287\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" Sep 15 07:31:47.235016 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.234820 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11660574-efac-4686-962b-934e3bd76287-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-cgt48\" (UID: \"11660574-efac-4686-962b-934e3bd76287\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" Sep 15 07:31:47.235605 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.235575 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11660574-efac-4686-962b-934e3bd76287-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-cgt48\" (UID: \"11660574-efac-4686-962b-934e3bd76287\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" Sep 15 07:31:47.253354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.253328 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pb29\" (UniqueName: \"kubernetes.io/projected/11660574-efac-4686-962b-934e3bd76287-kube-api-access-7pb29\") pod \"payload-processing-5b54b7fbb7-cgt48\" (UID: \"11660574-efac-4686-962b-934e3bd76287\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" Sep 15 07:31:47.328562 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.328529 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" Sep 15 07:31:47.596784 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:47.596761 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-cgt48"] Sep 15 07:31:47.599267 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:31:47.599230 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11660574_efac_4686_962b_934e3bd76287.slice/crio-fcbc660212c803bbe2535151c649e076d8184dd7d7827a9a69873e61e3d99186 WatchSource:0}: Error finding container fcbc660212c803bbe2535151c649e076d8184dd7d7827a9a69873e61e3d99186: Status 404 returned error can't find the container with id fcbc660212c803bbe2535151c649e076d8184dd7d7827a9a69873e61e3d99186 Sep 15 07:31:48.555883 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:48.555837 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" event={"ID":"11660574-efac-4686-962b-934e3bd76287","Type":"ContainerStarted","Data":"fcbc660212c803bbe2535151c649e076d8184dd7d7827a9a69873e61e3d99186"} Sep 15 07:31:49.561855 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:49.561814 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" event={"ID":"11660574-efac-4686-962b-934e3bd76287","Type":"ContainerStarted","Data":"9546210587e544df99213490f8ae370780ef5fe4cfa7050ba75021999fb22b24"} Sep 15 07:31:49.562349 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:49.561981 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" Sep 15 07:31:49.580927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:49.580878 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" podStartSLOduration=2.204925088 podStartE2EDuration="3.580852548s" totalImagesPullingTime="1.37592746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:31:49.5784541 +0000 UTC m=+790.559616267" watchObservedRunningTime="2026-09-15 07:31:49.580852548 +0000 UTC m=+790.562014777" Sep 15 07:31:56.517464 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:56.517430 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:31:56.607342 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:31:56.607302 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:32:00.569061 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:00.569031 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-cgt48" Sep 15 07:32:11.513816 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.513775 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx"] Sep 15 07:32:11.517120 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.517089 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.520506 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.520480 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 15 07:32:11.520646 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.520527 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 15 07:32:11.520646 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.520556 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-84rbz\"" Sep 15 07:32:11.520751 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.520680 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 15 07:32:11.528507 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.528483 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx"] Sep 15 07:32:11.545495 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.545466 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:32:11.645786 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.645748 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.645991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.645794 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.645991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.645953 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.645991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.645980 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vzjb\" (UniqueName: \"kubernetes.io/projected/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-kube-api-access-4vzjb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.646162 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.646001 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.646162 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.646093 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.747330 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.747285 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.747330 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.747327 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.747590 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.747348 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.747590 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.747408 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.747590 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.747425 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vzjb\" (UniqueName: \"kubernetes.io/projected/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-kube-api-access-4vzjb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.747590 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.747442 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.747788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.747714 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.747788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.747750 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.747918 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.747901 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.749695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.749671 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.749944 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.749927 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.755491 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.755469 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vzjb\" (UniqueName: \"kubernetes.io/projected/5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8-kube-api-access-4vzjb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx\" (UID: \"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:11.829435 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:11.829357 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:12.101136 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:12.101066 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx"] Sep 15 07:32:12.104496 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:32:12.104466 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0c1ffc_e0ee_4a6f_80b6_7051d0bc36a8.slice/crio-7e8f331ff6cfbbfe63412666b9c048dbf3970606fe41e0c56d21a9cfe46cb6bb WatchSource:0}: Error finding container 7e8f331ff6cfbbfe63412666b9c048dbf3970606fe41e0c56d21a9cfe46cb6bb: Status 404 returned error can't find the container with id 7e8f331ff6cfbbfe63412666b9c048dbf3970606fe41e0c56d21a9cfe46cb6bb Sep 15 07:32:12.664178 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:12.664137 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" event={"ID":"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8","Type":"ContainerStarted","Data":"7e8f331ff6cfbbfe63412666b9c048dbf3970606fe41e0c56d21a9cfe46cb6bb"} Sep 15 07:32:13.670404 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:13.670371 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" event={"ID":"5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8","Type":"ContainerStarted","Data":"79cdc4a33ba8a89574f3326a2ab6d3b5f3456400193a59e2f815ff54e044c8d1"} Sep 15 07:32:13.687639 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:13.687590 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" podStartSLOduration=1.65998819 podStartE2EDuration="2.687572507s" totalImagesPullingTime="1.027584317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:32:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:32:13.6868928 +0000 UTC m=+814.668054964" watchObservedRunningTime="2026-09-15 07:32:13.687572507 +0000 UTC m=+814.668734677" Sep 15 07:32:18.958111 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:18.958076 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:32:21.829642 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:21.829590 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:21.830080 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:21.829704 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:21.841943 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:21.841921 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:22.723713 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:22.723679 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx" Sep 15 07:32:41.814957 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:41.814925 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk"] Sep 15 07:32:41.817289 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:41.817273 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:41.819613 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:41.819592 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 15 07:32:41.826458 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:41.826433 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk"] Sep 15 07:32:41.901937 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:41.901903 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83b47a7d-a312-453b-8ebb-3e6a27b48459-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:41.902146 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:41.901951 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tvm2\" (UniqueName: \"kubernetes.io/projected/83b47a7d-a312-453b-8ebb-3e6a27b48459-kube-api-access-5tvm2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:41.902146 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:41.901983 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83b47a7d-a312-453b-8ebb-3e6a27b48459-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:41.902146 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:41.902039 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83b47a7d-a312-453b-8ebb-3e6a27b48459-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:41.902296 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:41.902158 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83b47a7d-a312-453b-8ebb-3e6a27b48459-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:41.902296 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:41.902196 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83b47a7d-a312-453b-8ebb-3e6a27b48459-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:41.905350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:41.905325 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:32:42.003588 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.003560 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83b47a7d-a312-453b-8ebb-3e6a27b48459-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.003588 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.003593 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tvm2\" (UniqueName: \"kubernetes.io/projected/83b47a7d-a312-453b-8ebb-3e6a27b48459-kube-api-access-5tvm2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.003778 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.003613 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83b47a7d-a312-453b-8ebb-3e6a27b48459-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.003778 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.003643 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83b47a7d-a312-453b-8ebb-3e6a27b48459-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.003778 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.003693 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83b47a7d-a312-453b-8ebb-3e6a27b48459-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.003778 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.003710 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83b47a7d-a312-453b-8ebb-3e6a27b48459-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.004134 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.004093 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83b47a7d-a312-453b-8ebb-3e6a27b48459-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.004203 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.004134 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83b47a7d-a312-453b-8ebb-3e6a27b48459-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.004203 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.004156 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83b47a7d-a312-453b-8ebb-3e6a27b48459-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.006068 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.006032 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83b47a7d-a312-453b-8ebb-3e6a27b48459-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.006213 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.006195 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83b47a7d-a312-453b-8ebb-3e6a27b48459-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.011647 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.011626 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tvm2\" (UniqueName: \"kubernetes.io/projected/83b47a7d-a312-453b-8ebb-3e6a27b48459-kube-api-access-5tvm2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk\" (UID: \"83b47a7d-a312-453b-8ebb-3e6a27b48459\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.128405 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.128306 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:42.397169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.397139 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk"] Sep 15 07:32:42.399546 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:32:42.399515 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83b47a7d_a312_453b_8ebb_3e6a27b48459.slice/crio-98334f158ffb4d67959d76bd1633e515c5c46f8bfc641833eb720f234a4895ef WatchSource:0}: Error finding container 98334f158ffb4d67959d76bd1633e515c5c46f8bfc641833eb720f234a4895ef: Status 404 returned error can't find the container with id 98334f158ffb4d67959d76bd1633e515c5c46f8bfc641833eb720f234a4895ef Sep 15 07:32:42.804359 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.804324 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" event={"ID":"83b47a7d-a312-453b-8ebb-3e6a27b48459","Type":"ContainerStarted","Data":"6b9f96dac19c41366db818148dac92e8f35b61b4eb2c9b6b7fa9963fac8f0133"} Sep 15 07:32:42.804560 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.804375 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" event={"ID":"83b47a7d-a312-453b-8ebb-3e6a27b48459","Type":"ContainerStarted","Data":"98334f158ffb4d67959d76bd1633e515c5c46f8bfc641833eb720f234a4895ef"} Sep 15 07:32:42.821942 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:42.821895 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" podStartSLOduration=1.6334833720000002 podStartE2EDuration="1.821854072s" totalImagesPullingTime="188.3707ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:32:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:32:42.820320282 +0000 UTC m=+843.801482449" watchObservedRunningTime="2026-09-15 07:32:42.821854072 +0000 UTC m=+843.803016238" Sep 15 07:32:52.128448 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:52.128409 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:52.128845 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:52.128472 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:52.141035 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:52.141009 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:32:52.860741 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:32:52.860712 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk" Sep 15 07:33:00.625521 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:00.625487 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:33:12.936731 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:12.936699 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:33:26.096529 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.096484 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv"] Sep 15 07:33:26.100043 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.100025 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.102416 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.102394 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 15 07:33:26.111120 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.111096 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv"] Sep 15 07:33:26.130827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.130799 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:33:26.183842 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.183815 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2988082b-93a8-48fd-bdfe-11977c4e016f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.183953 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.183857 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2988082b-93a8-48fd-bdfe-11977c4e016f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.183953 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.183935 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2988082b-93a8-48fd-bdfe-11977c4e016f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.184050 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.183990 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2988082b-93a8-48fd-bdfe-11977c4e016f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.184050 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.184013 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2988082b-93a8-48fd-bdfe-11977c4e016f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.184122 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.184069 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6fqx\" (UniqueName: \"kubernetes.io/projected/2988082b-93a8-48fd-bdfe-11977c4e016f-kube-api-access-p6fqx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.285243 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.285211 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6fqx\" (UniqueName: \"kubernetes.io/projected/2988082b-93a8-48fd-bdfe-11977c4e016f-kube-api-access-p6fqx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.285414 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.285250 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2988082b-93a8-48fd-bdfe-11977c4e016f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.285414 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.285286 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2988082b-93a8-48fd-bdfe-11977c4e016f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.285414 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.285313 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2988082b-93a8-48fd-bdfe-11977c4e016f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.285414 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.285372 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2988082b-93a8-48fd-bdfe-11977c4e016f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.285629 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.285598 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2988082b-93a8-48fd-bdfe-11977c4e016f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.285821 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.285802 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2988082b-93a8-48fd-bdfe-11977c4e016f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.285932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.285854 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2988082b-93a8-48fd-bdfe-11977c4e016f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.286010 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.285989 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2988082b-93a8-48fd-bdfe-11977c4e016f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.287628 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.287610 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2988082b-93a8-48fd-bdfe-11977c4e016f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.287771 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.287754 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2988082b-93a8-48fd-bdfe-11977c4e016f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.292597 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.292577 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6fqx\" (UniqueName: \"kubernetes.io/projected/2988082b-93a8-48fd-bdfe-11977c4e016f-kube-api-access-p6fqx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fkbwv\" (UID: \"2988082b-93a8-48fd-bdfe-11977c4e016f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.429379 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.429304 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:26.698400 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:26.698375 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv"] Sep 15 07:33:26.700778 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:33:26.700750 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2988082b_93a8_48fd_bdfe_11977c4e016f.slice/crio-040e0bf89f1a4580ab67e2cd92a385571171f55040ca1435293848e6410a624f WatchSource:0}: Error finding container 040e0bf89f1a4580ab67e2cd92a385571171f55040ca1435293848e6410a624f: Status 404 returned error can't find the container with id 040e0bf89f1a4580ab67e2cd92a385571171f55040ca1435293848e6410a624f Sep 15 07:33:27.004703 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:27.004666 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" event={"ID":"2988082b-93a8-48fd-bdfe-11977c4e016f","Type":"ContainerStarted","Data":"0e4f37e77a3162fe7728136044a502502c2bb39f9b116dfec4b18643d440f9a1"} Sep 15 07:33:27.004854 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:27.004724 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" event={"ID":"2988082b-93a8-48fd-bdfe-11977c4e016f","Type":"ContainerStarted","Data":"040e0bf89f1a4580ab67e2cd92a385571171f55040ca1435293848e6410a624f"} Sep 15 07:33:27.024124 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:27.024070 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" podStartSLOduration=0.829639462 podStartE2EDuration="1.024054866s" totalImagesPullingTime="194.415404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:33:27.022178842 +0000 UTC m=+888.003341012" watchObservedRunningTime="2026-09-15 07:33:27.024054866 +0000 UTC m=+888.005217033" Sep 15 07:33:36.430058 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:36.430015 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:36.430460 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:36.430104 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:36.442839 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:36.442816 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:37.060376 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:37.060348 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fkbwv" Sep 15 07:33:41.010538 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:41.010417 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:33:41.025861 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:41.012200 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:33:41.025861 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:41.017219 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:33:41.025861 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:41.018982 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:33:50.290541 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:33:50.290502 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:34:47.450207 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:34:47.450168 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:34:56.540654 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:34:56.540599 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:34:56.639620 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:34:56.639578 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:34:57.054608 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:34:57.054567 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:34:57.144385 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:34:57.144358 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:34:58.115419 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:34:58.115385 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:34:58.219505 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:34:58.219471 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:14.283379 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.283340 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:14.375756 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.375724 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:14.416306 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.416276 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q"] Sep 15 07:35:14.420150 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.420126 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.422545 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.422516 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-2735c5-openshift-default-dockercfg-dlvlf\"" Sep 15 07:35:14.432477 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.432452 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q"] Sep 15 07:35:14.446234 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.446206 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:14.525248 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.525213 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-podinfo\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.525448 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.525325 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-workload-socket\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.525448 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.525370 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istiod-ca-cert\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.525736 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.525709 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-workload-certs\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.525781 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.525739 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rx6b\" (UniqueName: \"kubernetes.io/projected/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-kube-api-access-4rx6b\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.525781 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.525762 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-token\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.525904 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.525862 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-envoy\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.526008 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.525991 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-credential-socket\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.526052 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.526040 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-data\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627167 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627094 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-workload-socket\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627167 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627129 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istiod-ca-cert\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627167 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627170 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-workload-certs\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627450 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627187 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rx6b\" (UniqueName: \"kubernetes.io/projected/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-kube-api-access-4rx6b\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627450 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627228 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-token\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627450 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627269 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-envoy\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627450 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627324 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-credential-socket\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627450 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627351 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-data\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627450 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627384 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-podinfo\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627753 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627504 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-workload-socket\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627753 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627557 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-workload-certs\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627891 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627821 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-credential-socket\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.627962 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.627920 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istiod-ca-cert\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.628022 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.628000 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-data\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.629744 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.629721 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-envoy\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.629959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.629943 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-podinfo\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.634807 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.634786 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-token\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.634972 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.634955 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rx6b\" (UniqueName: \"kubernetes.io/projected/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-kube-api-access-4rx6b\") pod \"e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:14.657194 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.657155 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:14.732678 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:14.732652 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:15.008088 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:15.008056 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q"] Sep 15 07:35:15.010205 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:35:15.010166 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfefaa36f_1ec1_4baf_83ef_f612b58b6cea.slice/crio-3ddb4f624f677dc9f7af0c35764d035cabd21485fc06296b30e5742bfca48544 WatchSource:0}: Error finding container 3ddb4f624f677dc9f7af0c35764d035cabd21485fc06296b30e5742bfca48544: Status 404 returned error can't find the container with id 3ddb4f624f677dc9f7af0c35764d035cabd21485fc06296b30e5742bfca48544 Sep 15 07:35:15.012456 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:15.012421 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:15.012545 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:15.012486 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:15.012545 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:15.012513 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:15.338597 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:15.338494 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:15.431860 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:15.431829 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:15.501291 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:15.501259 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" event={"ID":"fefaa36f-1ec1-4baf-83ef-f612b58b6cea","Type":"ContainerStarted","Data":"76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0"} Sep 15 07:35:15.501478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:15.501309 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" event={"ID":"fefaa36f-1ec1-4baf-83ef-f612b58b6cea","Type":"ContainerStarted","Data":"3ddb4f624f677dc9f7af0c35764d035cabd21485fc06296b30e5742bfca48544"} Sep 15 07:35:15.524754 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:15.524701 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" podStartSLOduration=1.5246855529999999 podStartE2EDuration="1.524685553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:35:15.522224307 +0000 UTC m=+996.503386480" watchObservedRunningTime="2026-09-15 07:35:15.524685553 +0000 UTC m=+996.505847720" Sep 15 07:35:15.733156 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:15.733131 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:15.754916 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:15.754861 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:16.737551 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:16.737520 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:17.436629 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:17.436590 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:17.512329 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:17.512298 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:17.513184 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:17.513163 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:17.530009 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:17.529978 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:17.634182 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:17.634149 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:17.729018 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:17.728990 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:18.943647 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:18.943613 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:19.043707 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.043665 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:19.161961 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.161931 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:19.591617 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.591590 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:19.693466 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.693434 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:19.742281 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.742249 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj"] Sep 15 07:35:19.746080 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.746057 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" Sep 15 07:35:19.748355 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.748335 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-2735c5-dockercfg-4lfvh\"" Sep 15 07:35:19.748480 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.748338 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-2735c5\"" Sep 15 07:35:19.756386 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.756360 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj"] Sep 15 07:35:19.883233 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.883145 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j58z\" (UniqueName: \"kubernetes.io/projected/f788e9db-15b8-4741-bd21-1c642563e556-kube-api-access-4j58z\") pod \"payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj\" (UID: \"f788e9db-15b8-4741-bd21-1c642563e556\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" Sep 15 07:35:19.883394 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.883318 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f788e9db-15b8-4741-bd21-1c642563e556-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj\" (UID: \"f788e9db-15b8-4741-bd21-1c642563e556\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" Sep 15 07:35:19.984479 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.984442 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f788e9db-15b8-4741-bd21-1c642563e556-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj\" (UID: \"f788e9db-15b8-4741-bd21-1c642563e556\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" Sep 15 07:35:19.984956 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.984495 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j58z\" (UniqueName: \"kubernetes.io/projected/f788e9db-15b8-4741-bd21-1c642563e556-kube-api-access-4j58z\") pod \"payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj\" (UID: \"f788e9db-15b8-4741-bd21-1c642563e556\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" Sep 15 07:35:19.985169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.985147 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f788e9db-15b8-4741-bd21-1c642563e556-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj\" (UID: \"f788e9db-15b8-4741-bd21-1c642563e556\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" Sep 15 07:35:19.993690 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:19.993667 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j58z\" (UniqueName: \"kubernetes.io/projected/f788e9db-15b8-4741-bd21-1c642563e556-kube-api-access-4j58z\") pod \"payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj\" (UID: \"f788e9db-15b8-4741-bd21-1c642563e556\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" Sep 15 07:35:20.058898 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:20.058857 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" Sep 15 07:35:20.157789 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:20.157715 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:20.213727 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:20.213694 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:20.329968 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:20.329910 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj"] Sep 15 07:35:20.332380 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:35:20.332355 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf788e9db_15b8_4741_bd21_1c642563e556.slice/crio-be309fddc1737dbd9b5b65b084d1d667bced446fec6ad61a48fb156bbbd0fe7c WatchSource:0}: Error finding container be309fddc1737dbd9b5b65b084d1d667bced446fec6ad61a48fb156bbbd0fe7c: Status 404 returned error can't find the container with id be309fddc1737dbd9b5b65b084d1d667bced446fec6ad61a48fb156bbbd0fe7c Sep 15 07:35:20.381315 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:20.381290 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:20.527666 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:20.527630 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" event={"ID":"f788e9db-15b8-4741-bd21-1c642563e556","Type":"ContainerStarted","Data":"1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a"} Sep 15 07:35:20.527826 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:20.527682 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" event={"ID":"f788e9db-15b8-4741-bd21-1c642563e556","Type":"ContainerStarted","Data":"be309fddc1737dbd9b5b65b084d1d667bced446fec6ad61a48fb156bbbd0fe7c"} Sep 15 07:35:20.527826 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:20.527703 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" Sep 15 07:35:20.545105 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:20.545055 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" podStartSLOduration=1.545040714 podStartE2EDuration="1.545040714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:35:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:35:20.5432765 +0000 UTC m=+1001.524438666" watchObservedRunningTime="2026-09-15 07:35:20.545040714 +0000 UTC m=+1001.526202878" Sep 15 07:35:20.640154 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:20.640117 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:20.738222 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:20.738191 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:21.052530 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:21.052494 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:21.154697 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:21.154661 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:21.253562 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:21.253532 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:21.567093 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:21.567063 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:21.876344 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:21.876244 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx"] Sep 15 07:35:21.880200 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:21.880174 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" Sep 15 07:35:21.884215 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:21.884177 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-ee0096-dockercfg-tsd4g\"" Sep 15 07:35:21.884360 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:21.884269 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-ee0096\"" Sep 15 07:35:21.895500 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:21.895475 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx"] Sep 15 07:35:22.002009 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.001970 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-959r6\" (UniqueName: \"kubernetes.io/projected/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c-kube-api-access-959r6\") pod \"payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx\" (UID: \"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" Sep 15 07:35:22.002172 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.002017 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx\" (UID: \"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" Sep 15 07:35:22.103290 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.103247 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-959r6\" (UniqueName: \"kubernetes.io/projected/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c-kube-api-access-959r6\") pod \"payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx\" (UID: \"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" Sep 15 07:35:22.103705 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.103299 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx\" (UID: \"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" Sep 15 07:35:22.104034 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.104017 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx\" (UID: \"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" Sep 15 07:35:22.117563 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.117539 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-959r6\" (UniqueName: \"kubernetes.io/projected/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c-kube-api-access-959r6\") pod \"payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx\" (UID: \"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" Sep 15 07:35:22.140436 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.140352 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg"] Sep 15 07:35:22.144675 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.144650 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.148910 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.148884 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-f33ec8-openshift-default-dockercfg-nq8fc\"" Sep 15 07:35:22.160561 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.160538 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg"] Sep 15 07:35:22.194297 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.194265 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" Sep 15 07:35:22.305315 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.305281 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-podinfo\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.305478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.305322 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-workload-socket\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.305478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.305343 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-token\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.305478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.305411 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-workload-certs\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.305478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.305449 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544eb4bf-b2a2-4376-9502-56aec29c9657-istiod-ca-cert\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.305721 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.305487 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-envoy\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.305721 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.305569 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-credential-socket\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.305721 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.305604 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hs52\" (UniqueName: \"kubernetes.io/projected/544eb4bf-b2a2-4376-9502-56aec29c9657-kube-api-access-2hs52\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.305721 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.305628 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-data\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407092 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407008 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-podinfo\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407092 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407063 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-workload-socket\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407094 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-token\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407123 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-workload-certs\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407146 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544eb4bf-b2a2-4376-9502-56aec29c9657-istiod-ca-cert\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407181 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-envoy\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407250 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-credential-socket\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407286 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hs52\" (UniqueName: \"kubernetes.io/projected/544eb4bf-b2a2-4376-9502-56aec29c9657-kube-api-access-2hs52\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407311 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-data\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407713 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407507 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-workload-socket\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407713 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407655 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-data\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407831 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407773 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-credential-socket\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.407962 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.407938 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544eb4bf-b2a2-4376-9502-56aec29c9657-istiod-ca-cert\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.408213 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.408197 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-workload-certs\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.410015 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.409988 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-envoy\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.410166 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.410149 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-podinfo\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.414707 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.414677 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-token\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.414844 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.414827 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hs52\" (UniqueName: \"kubernetes.io/projected/544eb4bf-b2a2-4376-9502-56aec29c9657-kube-api-access-2hs52\") pod \"e2e-autores-f33ec8-openshift-default-684b546598-5tzpg\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.460562 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.460537 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:22.468330 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.468310 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx"] Sep 15 07:35:22.471434 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:35:22.471406 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a9aa094_c37b_4f88_a7fc_4a27355cfe8c.slice/crio-cfb79d0296497213ddbc871c4b793b5bd18e272589ba9fc69f7be5b4958020a8 WatchSource:0}: Error finding container cfb79d0296497213ddbc871c4b793b5bd18e272589ba9fc69f7be5b4958020a8: Status 404 returned error can't find the container with id cfb79d0296497213ddbc871c4b793b5bd18e272589ba9fc69f7be5b4958020a8 Sep 15 07:35:22.543282 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.543241 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" event={"ID":"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c","Type":"ContainerStarted","Data":"cfb79d0296497213ddbc871c4b793b5bd18e272589ba9fc69f7be5b4958020a8"} Sep 15 07:35:22.751001 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.750975 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg"] Sep 15 07:35:22.753178 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:35:22.753148 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod544eb4bf_b2a2_4376_9502_56aec29c9657.slice/crio-76ceb3ff0393501a38849a147a973227ba7ab07bedf6deaba0d23d7b6e13cb87 WatchSource:0}: Error finding container 76ceb3ff0393501a38849a147a973227ba7ab07bedf6deaba0d23d7b6e13cb87: Status 404 returned error can't find the container with id 76ceb3ff0393501a38849a147a973227ba7ab07bedf6deaba0d23d7b6e13cb87 Sep 15 07:35:22.755411 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.755382 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:22.755500 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.755445 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:22.755500 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:22.755474 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:23.549730 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:23.549693 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" event={"ID":"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c","Type":"ContainerStarted","Data":"63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30"} Sep 15 07:35:23.550212 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:23.549803 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" Sep 15 07:35:23.551379 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:23.551356 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" event={"ID":"544eb4bf-b2a2-4376-9502-56aec29c9657","Type":"ContainerStarted","Data":"efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd"} Sep 15 07:35:23.551511 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:23.551390 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" event={"ID":"544eb4bf-b2a2-4376-9502-56aec29c9657","Type":"ContainerStarted","Data":"76ceb3ff0393501a38849a147a973227ba7ab07bedf6deaba0d23d7b6e13cb87"} Sep 15 07:35:23.571362 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:23.571306 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" podStartSLOduration=2.5712892309999997 podStartE2EDuration="2.571289231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:35:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:35:23.568880809 +0000 UTC m=+1004.550042973" watchObservedRunningTime="2026-09-15 07:35:23.571289231 +0000 UTC m=+1004.552451418" Sep 15 07:35:23.588349 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:23.588301 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" podStartSLOduration=1.58828761 podStartE2EDuration="1.58828761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:35:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:35:23.586884369 +0000 UTC m=+1004.568046537" watchObservedRunningTime="2026-09-15 07:35:23.58828761 +0000 UTC m=+1004.569449777" Sep 15 07:35:24.149943 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:24.149904 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:24.249497 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:24.249463 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:24.460860 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:24.460767 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:24.465743 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:24.465719 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:24.558552 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:24.558519 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:24.559442 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:24.559418 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:25.451061 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:25.451019 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:25.551779 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:25.551745 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:25.653211 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:25.653179 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:26.069370 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:26.069338 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:26.165547 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:26.165516 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:26.633451 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:26.633422 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:27.825585 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:27.825552 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:28.905575 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:28.905540 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:29.007513 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:29.007462 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:29.152788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:29.152752 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:29.462903 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:29.462846 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:29.562618 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:29.562587 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:30.012442 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:30.012399 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:31.266665 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:31.266628 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:31.342178 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:31.342139 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:31.427330 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:31.427293 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:31.534731 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:31.534649 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" Sep 15 07:35:31.936402 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:31.936309 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:32.039339 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.039299 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:32.190745 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.190652 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462"] Sep 15 07:35:32.204139 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.204109 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462"] Sep 15 07:35:32.204378 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.204189 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" Sep 15 07:35:32.207044 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.207024 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-f33ec8\"" Sep 15 07:35:32.207199 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.207062 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-f33ec8-dockercfg-b9576\"" Sep 15 07:35:32.226977 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.226947 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08c58bcf-9c79-49af-ae5c-47ba44b28daf-plugins-config-volume\") pod \"payload-processing-e2e-autores-f33ec8-84f48486c7-bx462\" (UID: \"08c58bcf-9c79-49af-ae5c-47ba44b28daf\") " pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" Sep 15 07:35:32.227099 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.227031 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj6m6\" (UniqueName: \"kubernetes.io/projected/08c58bcf-9c79-49af-ae5c-47ba44b28daf-kube-api-access-bj6m6\") pod \"payload-processing-e2e-autores-f33ec8-84f48486c7-bx462\" (UID: \"08c58bcf-9c79-49af-ae5c-47ba44b28daf\") " pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" Sep 15 07:35:32.327586 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.327543 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj6m6\" (UniqueName: \"kubernetes.io/projected/08c58bcf-9c79-49af-ae5c-47ba44b28daf-kube-api-access-bj6m6\") pod \"payload-processing-e2e-autores-f33ec8-84f48486c7-bx462\" (UID: \"08c58bcf-9c79-49af-ae5c-47ba44b28daf\") " pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" Sep 15 07:35:32.328015 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.327660 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08c58bcf-9c79-49af-ae5c-47ba44b28daf-plugins-config-volume\") pod \"payload-processing-e2e-autores-f33ec8-84f48486c7-bx462\" (UID: \"08c58bcf-9c79-49af-ae5c-47ba44b28daf\") " pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" Sep 15 07:35:32.328410 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.328391 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08c58bcf-9c79-49af-ae5c-47ba44b28daf-plugins-config-volume\") pod \"payload-processing-e2e-autores-f33ec8-84f48486c7-bx462\" (UID: \"08c58bcf-9c79-49af-ae5c-47ba44b28daf\") " pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" Sep 15 07:35:32.335656 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.335631 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj6m6\" (UniqueName: \"kubernetes.io/projected/08c58bcf-9c79-49af-ae5c-47ba44b28daf-kube-api-access-bj6m6\") pod \"payload-processing-e2e-autores-f33ec8-84f48486c7-bx462\" (UID: \"08c58bcf-9c79-49af-ae5c-47ba44b28daf\") " pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" Sep 15 07:35:32.337577 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.337550 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:32.479465 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.479434 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:32.516682 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.516649 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" Sep 15 07:35:32.689892 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.688338 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:32.813908 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.812650 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462"] Sep 15 07:35:32.814356 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:35:32.814228 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c58bcf_9c79_49af_ae5c_47ba44b28daf.slice/crio-92bc98d229bf0c8ffd73321c2ba3e6da82eabb74da0702bfed80d35e6f017285 WatchSource:0}: Error finding container 92bc98d229bf0c8ffd73321c2ba3e6da82eabb74da0702bfed80d35e6f017285: Status 404 returned error can't find the container with id 92bc98d229bf0c8ffd73321c2ba3e6da82eabb74da0702bfed80d35e6f017285 Sep 15 07:35:32.975889 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:32.975583 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r"] Sep 15 07:35:33.001910 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.001632 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.004612 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.004583 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r"] Sep 15 07:35:33.005363 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.005170 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-fcd453-openshift-default-dockercfg-zvmnt\"" Sep 15 07:35:33.034279 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.034253 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff0ba423-6d42-423e-a11b-327dca1e6935-istiod-ca-cert\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.034415 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.034299 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-workload-socket\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.034900 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.034535 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-data\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.034986 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.034945 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-podinfo\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.034986 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.034973 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-envoy\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.035119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.035086 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-token\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.035395 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.035196 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-workload-certs\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.035395 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.035292 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msfk4\" (UniqueName: \"kubernetes.io/projected/ff0ba423-6d42-423e-a11b-327dca1e6935-kube-api-access-msfk4\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.035395 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.035322 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-credential-socket\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.136717 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.136613 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msfk4\" (UniqueName: \"kubernetes.io/projected/ff0ba423-6d42-423e-a11b-327dca1e6935-kube-api-access-msfk4\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.136717 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.136660 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-credential-socket\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.136717 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.136716 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff0ba423-6d42-423e-a11b-327dca1e6935-istiod-ca-cert\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.137069 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.136747 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-workload-socket\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.137069 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.136795 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-data\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.137069 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.136832 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-podinfo\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.137069 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.136854 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-envoy\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.137069 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.136930 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-token\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.137069 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.136960 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-workload-certs\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.137380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.137197 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-credential-socket\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.137380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.137247 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-workload-socket\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.137599 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.137376 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-workload-certs\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.137832 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.137789 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-data\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.138102 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.138079 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff0ba423-6d42-423e-a11b-327dca1e6935-istiod-ca-cert\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.139668 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.139648 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-envoy\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.139961 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.139939 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-podinfo\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.146138 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.146110 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-token\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.146472 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.146430 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msfk4\" (UniqueName: \"kubernetes.io/projected/ff0ba423-6d42-423e-a11b-327dca1e6935-kube-api-access-msfk4\") pod \"e2e-inf-b-fcd453-openshift-default-6758677554-78r2r\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.314600 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.314412 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:33.526763 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.526725 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:33.599440 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.599401 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r"] Sep 15 07:35:33.600666 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.600590 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:33.601569 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:35:33.601544 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0ba423_6d42_423e_a11b_327dca1e6935.slice/crio-2123cec6187eca9ba00e641818b6ec92468c908c88ec3bfeb3016b6f3fe5948e WatchSource:0}: Error finding container 2123cec6187eca9ba00e641818b6ec92468c908c88ec3bfeb3016b6f3fe5948e: Status 404 returned error can't find the container with id 2123cec6187eca9ba00e641818b6ec92468c908c88ec3bfeb3016b6f3fe5948e Sep 15 07:35:33.604482 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.604447 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:33.604574 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.604529 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:33.604574 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.604567 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:33.617150 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.617122 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" event={"ID":"08c58bcf-9c79-49af-ae5c-47ba44b28daf","Type":"ContainerStarted","Data":"d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a"} Sep 15 07:35:33.617261 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.617186 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" Sep 15 07:35:33.617261 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.617211 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" event={"ID":"08c58bcf-9c79-49af-ae5c-47ba44b28daf","Type":"ContainerStarted","Data":"92bc98d229bf0c8ffd73321c2ba3e6da82eabb74da0702bfed80d35e6f017285"} Sep 15 07:35:33.618629 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.618604 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" event={"ID":"ff0ba423-6d42-423e-a11b-327dca1e6935","Type":"ContainerStarted","Data":"2123cec6187eca9ba00e641818b6ec92468c908c88ec3bfeb3016b6f3fe5948e"} Sep 15 07:35:33.636033 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.635986 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" podStartSLOduration=1.635970176 podStartE2EDuration="1.635970176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:35:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:35:33.633526899 +0000 UTC m=+1014.614689068" watchObservedRunningTime="2026-09-15 07:35:33.635970176 +0000 UTC m=+1014.617132343" Sep 15 07:35:33.660448 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.660412 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:33.761389 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:33.761348 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:34.260652 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:34.260614 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:34.560655 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:34.560581 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" Sep 15 07:35:34.629944 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:34.629911 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" event={"ID":"ff0ba423-6d42-423e-a11b-327dca1e6935","Type":"ContainerStarted","Data":"23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7"} Sep 15 07:35:34.648433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:34.648384 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" podStartSLOduration=2.64836635 podStartE2EDuration="2.64836635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:35:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:35:34.647192393 +0000 UTC m=+1015.628354554" watchObservedRunningTime="2026-09-15 07:35:34.64836635 +0000 UTC m=+1015.629528516" Sep 15 07:35:35.315238 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:35.315205 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:35.320244 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:35.320218 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:35.635646 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:35.635566 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:35.636552 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:35.636530 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:35:36.642028 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:36.641932 2552 generic.go:356] "Generic (PLEG): container finished" podID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerID="76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0" exitCode=0 Sep 15 07:35:36.642028 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:36.642002 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" event={"ID":"fefaa36f-1ec1-4baf-83ef-f612b58b6cea","Type":"ContainerDied","Data":"76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0"} Sep 15 07:35:36.642511 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:36.642493 2552 scope.go:117] "RemoveContainer" containerID="76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0" Sep 15 07:35:36.643725 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:36.643700 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:36.643784 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:36.643770 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:36.643823 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:36.643801 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:37.461110 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:37.461018 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:37.555693 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:37.555651 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:37.648325 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:37.648291 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" event={"ID":"fefaa36f-1ec1-4baf-83ef-f612b58b6cea","Type":"ContainerStarted","Data":"90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f"} Sep 15 07:35:37.733037 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:37.732998 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:37.733225 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:37.733199 2552 patch_prober.go:28] interesting pod/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Sep 15 07:35:37.733272 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:37.733252 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Sep 15 07:35:37.738858 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:37.738828 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:37.837259 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:37.837221 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:37.907574 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:37.907518 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:38.733605 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:38.733569 2552 patch_prober.go:28] interesting pod/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Sep 15 07:35:38.734025 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:38.733622 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Sep 15 07:35:38.744188 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:38.744148 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:38.848234 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:38.848202 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:38.977533 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:38.977470 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:39.384558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.384455 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:39.540883 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.540848 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:39.564143 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.564115 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945"] Sep 15 07:35:39.567846 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.567830 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" Sep 15 07:35:39.575200 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.575174 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-fcd453-dockercfg-phkk9\"" Sep 15 07:35:39.575673 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.575653 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-fcd453\"" Sep 15 07:35:39.593126 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.592996 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945"] Sep 15 07:35:39.611603 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.611561 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/868173b2-89ca-4d72-8d5b-c35440bbdd92-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945\" (UID: \"868173b2-89ca-4d72-8d5b-c35440bbdd92\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" Sep 15 07:35:39.611791 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.611632 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7zft\" (UniqueName: \"kubernetes.io/projected/868173b2-89ca-4d72-8d5b-c35440bbdd92-kube-api-access-k7zft\") pod \"payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945\" (UID: \"868173b2-89ca-4d72-8d5b-c35440bbdd92\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" Sep 15 07:35:39.713263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.713185 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/868173b2-89ca-4d72-8d5b-c35440bbdd92-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945\" (UID: \"868173b2-89ca-4d72-8d5b-c35440bbdd92\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" Sep 15 07:35:39.713263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.713250 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7zft\" (UniqueName: \"kubernetes.io/projected/868173b2-89ca-4d72-8d5b-c35440bbdd92-kube-api-access-k7zft\") pod \"payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945\" (UID: \"868173b2-89ca-4d72-8d5b-c35440bbdd92\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" Sep 15 07:35:39.713907 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.713882 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/868173b2-89ca-4d72-8d5b-c35440bbdd92-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945\" (UID: \"868173b2-89ca-4d72-8d5b-c35440bbdd92\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" Sep 15 07:35:39.726403 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.726378 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7zft\" (UniqueName: \"kubernetes.io/projected/868173b2-89ca-4d72-8d5b-c35440bbdd92-kube-api-access-k7zft\") pod \"payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945\" (UID: \"868173b2-89ca-4d72-8d5b-c35440bbdd92\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" Sep 15 07:35:39.733375 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.733350 2552 patch_prober.go:28] interesting pod/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Sep 15 07:35:39.733503 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.733407 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Sep 15 07:35:39.902101 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:39.902060 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" Sep 15 07:35:40.154292 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:40.154260 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:40.193711 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:40.193680 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945"] Sep 15 07:35:40.197856 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:35:40.197823 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod868173b2_89ca_4d72_8d5b_c35440bbdd92.slice/crio-19653a11ee2aee3fd8ee758c6d8da440be66eaa4802d36446e37c80221df6faa WatchSource:0}: Error finding container 19653a11ee2aee3fd8ee758c6d8da440be66eaa4802d36446e37c80221df6faa: Status 404 returned error can't find the container with id 19653a11ee2aee3fd8ee758c6d8da440be66eaa4802d36446e37c80221df6faa Sep 15 07:35:40.666588 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:40.666557 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" event={"ID":"868173b2-89ca-4d72-8d5b-c35440bbdd92","Type":"ContainerStarted","Data":"7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69"} Sep 15 07:35:40.666777 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:40.666607 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" event={"ID":"868173b2-89ca-4d72-8d5b-c35440bbdd92","Type":"ContainerStarted","Data":"19653a11ee2aee3fd8ee758c6d8da440be66eaa4802d36446e37c80221df6faa"} Sep 15 07:35:40.666777 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:40.666649 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" Sep 15 07:35:40.733860 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:40.733823 2552 patch_prober.go:28] interesting pod/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Sep 15 07:35:40.734081 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:40.733900 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Sep 15 07:35:41.225578 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:41.225539 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:41.557135 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:41.557037 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:41.733753 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:41.733719 2552 patch_prober.go:28] interesting pod/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Sep 15 07:35:41.733955 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:41.733778 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Sep 15 07:35:42.841325 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:42.841288 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:42.841818 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:42.841776 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:42.843083 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:42.843062 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:35:42.867228 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:42.867166 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" podStartSLOduration=3.867146067 podStartE2EDuration="3.867146067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:35:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:35:40.686313345 +0000 UTC m=+1021.667475513" watchObservedRunningTime="2026-09-15 07:35:42.867146067 +0000 UTC m=+1023.848308236" Sep 15 07:35:44.604518 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:44.604474 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:44.632045 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:44.632017 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" Sep 15 07:35:46.431596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:46.431559 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p"] Sep 15 07:35:46.435037 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:46.435012 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p" Sep 15 07:35:46.437226 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:46.437203 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 07:35:46.437354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:46.437207 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8jpwk\"" Sep 15 07:35:46.441625 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:46.441600 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p"] Sep 15 07:35:46.503122 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:46.503084 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca324875-07b8-4652-aa51-a9e5d1403081-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p\" (UID: \"ca324875-07b8-4652-aa51-a9e5d1403081\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p" Sep 15 07:35:46.604727 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:46.604688 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca324875-07b8-4652-aa51-a9e5d1403081-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p\" (UID: \"ca324875-07b8-4652-aa51-a9e5d1403081\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p" Sep 15 07:35:46.605457 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:46.605431 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca324875-07b8-4652-aa51-a9e5d1403081-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p\" (UID: \"ca324875-07b8-4652-aa51-a9e5d1403081\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p" Sep 15 07:35:46.747975 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:46.747946 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p" Sep 15 07:35:46.799615 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:46.799582 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:46.888420 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:46.888343 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:47.045975 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:47.045950 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p"] Sep 15 07:35:47.048692 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:35:47.048660 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca324875_07b8_4652_aa51_a9e5d1403081.slice/crio-d8e708d6578870648ca247a8a6d3a72440f7f38f85a94561a03b3cbf854428a7 WatchSource:0}: Error finding container d8e708d6578870648ca247a8a6d3a72440f7f38f85a94561a03b3cbf854428a7: Status 404 returned error can't find the container with id d8e708d6578870648ca247a8a6d3a72440f7f38f85a94561a03b3cbf854428a7 Sep 15 07:35:47.331500 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:47.331415 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:47.722034 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:47.721947 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p" event={"ID":"ca324875-07b8-4652-aa51-a9e5d1403081","Type":"ContainerStarted","Data":"d8e708d6578870648ca247a8a6d3a72440f7f38f85a94561a03b3cbf854428a7"} Sep 15 07:35:48.162624 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:48.162591 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:50.683020 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:50.682986 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:50.748606 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:50.748543 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:50.956201 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:50.956111 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:51.233411 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:51.233374 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:51.678529 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:51.678448 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" Sep 15 07:35:51.685652 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:51.685609 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:51.745942 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:51.745835 2552 generic.go:356] "Generic (PLEG): container finished" podID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerID="efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd" exitCode=0 Sep 15 07:35:51.745942 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:51.745903 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" event={"ID":"544eb4bf-b2a2-4376-9502-56aec29c9657","Type":"ContainerDied","Data":"efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd"} Sep 15 07:35:51.746336 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:51.746322 2552 scope.go:117] "RemoveContainer" containerID="efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd" Sep 15 07:35:51.747687 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:51.747657 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:51.747805 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:51.747741 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:51.747805 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:51.747788 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:35:52.432757 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:52.432722 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:52.754103 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:52.754067 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" event={"ID":"544eb4bf-b2a2-4376-9502-56aec29c9657","Type":"ContainerStarted","Data":"f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed"} Sep 15 07:35:53.461500 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:53.461468 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:53.462630 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:53.462601 2552 patch_prober.go:28] interesting pod/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Sep 15 07:35:53.462688 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:53.462651 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Sep 15 07:35:53.760435 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:53.760392 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p" event={"ID":"ca324875-07b8-4652-aa51-a9e5d1403081","Type":"ContainerStarted","Data":"e9a06b51d4b9ad9f9af9e4dafd2c3d658cd171fea2f37ab81d54622b0807e581"} Sep 15 07:35:53.790455 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:53.790390 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p" podStartSLOduration=1.559287618 podStartE2EDuration="7.790366779s" totalImagesPullingTime="6.231079161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:35:53.786974399 +0000 UTC m=+1034.768136569" watchObservedRunningTime="2026-09-15 07:35:53.790366779 +0000 UTC m=+1034.771528947" Sep 15 07:35:54.277913 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:54.277778 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:54.381894 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:54.379364 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:35:54.461515 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:54.461475 2552 patch_prober.go:28] interesting pod/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Sep 15 07:35:54.461668 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:54.461545 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Sep 15 07:35:54.558690 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:54.558606 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:54.766817 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:54.766783 2552 generic.go:356] "Generic (PLEG): container finished" podID="ca324875-07b8-4652-aa51-a9e5d1403081" containerID="e9a06b51d4b9ad9f9af9e4dafd2c3d658cd171fea2f37ab81d54622b0807e581" exitCode=0 Sep 15 07:35:54.767216 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:54.766861 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p" event={"ID":"ca324875-07b8-4652-aa51-a9e5d1403081","Type":"ContainerDied","Data":"e9a06b51d4b9ad9f9af9e4dafd2c3d658cd171fea2f37ab81d54622b0807e581"} Sep 15 07:35:55.461106 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:55.461070 2552 patch_prober.go:28] interesting pod/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Sep 15 07:35:55.461292 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:55.461134 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Sep 15 07:35:56.022725 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:56.022692 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p" Sep 15 07:35:56.124600 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:56.124509 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca324875-07b8-4652-aa51-a9e5d1403081-maas-api-service-ca\") pod \"ca324875-07b8-4652-aa51-a9e5d1403081\" (UID: \"ca324875-07b8-4652-aa51-a9e5d1403081\") " Sep 15 07:35:56.125018 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:56.124998 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca324875-07b8-4652-aa51-a9e5d1403081-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ca324875-07b8-4652-aa51-a9e5d1403081" (UID: "ca324875-07b8-4652-aa51-a9e5d1403081"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:35:56.225570 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:56.225529 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca324875-07b8-4652-aa51-a9e5d1403081-maas-api-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:35:56.461816 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:56.461733 2552 patch_prober.go:28] interesting pod/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Sep 15 07:35:56.461816 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:56.461797 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Sep 15 07:35:56.784260 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:56.784218 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p" event={"ID":"ca324875-07b8-4652-aa51-a9e5d1403081","Type":"ContainerDied","Data":"d8e708d6578870648ca247a8a6d3a72440f7f38f85a94561a03b3cbf854428a7"} Sep 15 07:35:56.784471 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:56.784284 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d8e708d6578870648ca247a8a6d3a72440f7f38f85a94561a03b3cbf854428a7" Sep 15 07:35:56.784471 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:56.784353 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p" Sep 15 07:35:57.461924 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:57.461891 2552 patch_prober.go:28] interesting pod/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Sep 15 07:35:57.462315 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:57.461947 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Sep 15 07:35:58.461372 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:58.461328 2552 patch_prober.go:28] interesting pod/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Sep 15 07:35:58.461564 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:58.461389 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Sep 15 07:35:59.465583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:59.465547 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:35:59.466589 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:35:59.466568 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:36:00.204962 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:00.204923 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:00.373513 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:00.373480 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:00.845061 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:00.845019 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:00.946339 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:00.946257 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:03.822855 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:03.822824 2552 generic.go:356] "Generic (PLEG): container finished" podID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerID="23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7" exitCode=0 Sep 15 07:36:03.823321 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:03.822908 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" event={"ID":"ff0ba423-6d42-423e-a11b-327dca1e6935","Type":"ContainerDied","Data":"23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7"} Sep 15 07:36:03.823372 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:03.823326 2552 scope.go:117] "RemoveContainer" containerID="23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7" Sep 15 07:36:03.824474 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:03.824448 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:36:03.824534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:03.824514 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:36:03.824570 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:03.824543 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:36:04.830845 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:04.830810 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" event={"ID":"ff0ba423-6d42-423e-a11b-327dca1e6935","Type":"ContainerStarted","Data":"c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858"} Sep 15 07:36:04.899785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:04.899747 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:05.314937 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:05.314901 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:36:05.316159 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:05.316131 2552 patch_prober.go:28] interesting pod/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 15 07:36:05.316288 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:05.316181 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 15 07:36:05.635897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:05.635784 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:36:06.315704 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:06.315667 2552 patch_prober.go:28] interesting pod/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 15 07:36:06.316133 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:06.315721 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 15 07:36:06.593931 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:06.593826 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:06.655663 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:06.655601 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:06.744332 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:06.744297 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:06.861198 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:06.861105 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:06.970518 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:06.970485 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:07.315058 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:07.315027 2552 patch_prober.go:28] interesting pod/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 15 07:36:07.315235 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:07.315078 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 15 07:36:07.449222 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:07.449186 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:07.633076 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:07.632992 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p"] Sep 15 07:36:07.635237 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:07.635208 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e8faa2dc-68g4p"] Sep 15 07:36:08.315649 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:08.315603 2552 patch_prober.go:28] interesting pod/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 15 07:36:08.315824 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:08.315688 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 15 07:36:08.766503 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:08.766415 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:08.866173 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:08.866119 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:08.984495 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:08.984454 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:09.116153 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:09.116051 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:09.315678 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:09.315641 2552 patch_prober.go:28] interesting pod/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 15 07:36:09.315855 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:09.315694 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 15 07:36:09.498985 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:09.498948 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca324875-07b8-4652-aa51-a9e5d1403081" path="/var/lib/kubelet/pods/ca324875-07b8-4652-aa51-a9e5d1403081/volumes" Sep 15 07:36:09.720375 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:09.720341 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:10.315008 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:10.314965 2552 patch_prober.go:28] interesting pod/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Sep 15 07:36:10.315517 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:10.315017 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Sep 15 07:36:10.634969 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:10.634849 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:11.320077 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:11.320046 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:36:11.321173 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:11.321151 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:36:11.401816 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:11.401779 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:11.510457 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:11.510421 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:13.576052 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:13.576012 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:13.649011 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:13.648975 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:14.043416 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:14.043387 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:14.853767 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:14.853736 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:15.653916 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:15.653852 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:16.364024 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:16.363984 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:17.228157 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:17.228125 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:31.711250 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:31.711210 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:35.363255 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:35.363218 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:36.366071 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.366036 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:36.426576 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.426547 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7"] Sep 15 07:36:36.426965 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.426951 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca324875-07b8-4652-aa51-a9e5d1403081" containerName="revoke-keys" Sep 15 07:36:36.426965 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.426966 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca324875-07b8-4652-aa51-a9e5d1403081" containerName="revoke-keys" Sep 15 07:36:36.427093 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.427035 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca324875-07b8-4652-aa51-a9e5d1403081" containerName="revoke-keys" Sep 15 07:36:36.432425 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.432388 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7" Sep 15 07:36:36.435114 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.435092 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 07:36:36.435618 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.435092 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8jpwk\"" Sep 15 07:36:36.442551 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.442524 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7"] Sep 15 07:36:36.533554 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.533522 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7\" (UID: \"2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7" Sep 15 07:36:36.634813 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.634735 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7\" (UID: \"2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7" Sep 15 07:36:36.635348 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.635327 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7\" (UID: \"2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7" Sep 15 07:36:36.753539 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:36.753508 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7" Sep 15 07:36:37.023018 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:37.022982 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7"] Sep 15 07:36:37.025317 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:36:37.025289 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e5b9a53_50d4_4282_aaa8_2b52e31e4ceb.slice/crio-ca807db8d077fcc226eda97d9bb6fb9c779880908d897e99a2349ebea075a0fb WatchSource:0}: Error finding container ca807db8d077fcc226eda97d9bb6fb9c779880908d897e99a2349ebea075a0fb: Status 404 returned error can't find the container with id ca807db8d077fcc226eda97d9bb6fb9c779880908d897e99a2349ebea075a0fb Sep 15 07:36:37.261804 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:37.261776 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:38.003026 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:38.002995 2552 generic.go:356] "Generic (PLEG): container finished" podID="2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb" containerID="58d9a2df86dcab772ae4f08f038b309ddd3e00f232090ccca9872c75ed7c8b4b" exitCode=0 Sep 15 07:36:38.003406 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:38.003078 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7" event={"ID":"2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb","Type":"ContainerDied","Data":"58d9a2df86dcab772ae4f08f038b309ddd3e00f232090ccca9872c75ed7c8b4b"} Sep 15 07:36:38.003406 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:38.003127 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7" event={"ID":"2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb","Type":"ContainerStarted","Data":"ca807db8d077fcc226eda97d9bb6fb9c779880908d897e99a2349ebea075a0fb"} Sep 15 07:36:38.060665 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:38.060637 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:38.970242 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:38.970206 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:39.170198 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:39.170177 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7" Sep 15 07:36:39.260211 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:39.260182 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb-maas-api-service-ca\") pod \"2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb\" (UID: \"2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb\") " Sep 15 07:36:39.260499 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:39.260476 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb" (UID: "2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:36:39.361719 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:39.361690 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb-maas-api-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:36:39.847439 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:39.847404 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:40.014566 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:40.014531 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7" Sep 15 07:36:40.014731 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:40.014537 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7" event={"ID":"2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb","Type":"ContainerDied","Data":"ca807db8d077fcc226eda97d9bb6fb9c779880908d897e99a2349ebea075a0fb"} Sep 15 07:36:40.014731 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:40.014657 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca807db8d077fcc226eda97d9bb6fb9c779880908d897e99a2349ebea075a0fb" Sep 15 07:36:40.767908 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:40.767860 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:41.412805 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:41.412773 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:42.088282 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:42.088250 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:42.695507 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:42.695462 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:42.770555 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:42.770522 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:42.827627 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:42.827594 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:42.896625 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:42.896587 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7"] Sep 15 07:36:42.898746 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:42.898698 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:42.899020 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:42.899001 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4e9b0a83-bt9h7"] Sep 15 07:36:42.955899 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:42.955793 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:43.042175 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.042147 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:43.351830 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.351797 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:43.475612 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.475584 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb" path="/var/lib/kubelet/pods/2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb/volumes" Sep 15 07:36:43.491697 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.491656 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:43.758019 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.757985 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js"] Sep 15 07:36:43.758598 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.758582 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb" containerName="revoke-keys" Sep 15 07:36:43.758651 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.758602 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb" containerName="revoke-keys" Sep 15 07:36:43.758737 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.758725 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e5b9a53-50d4-4282-aaa8-2b52e31e4ceb" containerName="revoke-keys" Sep 15 07:36:43.762961 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.762893 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.765884 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.765847 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b63bda62-openshift-default-dockercfg-k2qnm\"" Sep 15 07:36:43.775081 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.775060 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js"] Sep 15 07:36:43.796938 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.796917 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:43.806784 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.806758 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-token\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.806926 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.806889 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-credential-socket\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.806992 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.806946 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-data\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.806992 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.806976 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-workload-certs\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.807100 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.806994 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpp2n\" (UniqueName: \"kubernetes.io/projected/807e1e44-e04f-4ee5-93ac-8450a1b75529-kube-api-access-gpp2n\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.807157 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.807105 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-workload-socket\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.807210 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.807187 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-envoy\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.807263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.807253 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-podinfo\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.807328 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.807289 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/807e1e44-e04f-4ee5-93ac-8450a1b75529-istiod-ca-cert\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908058 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908009 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-credential-socket\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908058 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908066 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-data\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908325 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908089 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-workload-certs\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908325 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908105 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpp2n\" (UniqueName: \"kubernetes.io/projected/807e1e44-e04f-4ee5-93ac-8450a1b75529-kube-api-access-gpp2n\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908325 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908161 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-workload-socket\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908325 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908210 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-envoy\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908325 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908265 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-podinfo\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908325 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908294 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/807e1e44-e04f-4ee5-93ac-8450a1b75529-istiod-ca-cert\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908325 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908318 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-token\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908727 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908467 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-credential-socket\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908727 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908487 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-workload-certs\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908849 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908828 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-workload-socket\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.908936 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.908880 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-data\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.909215 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.909196 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/807e1e44-e04f-4ee5-93ac-8450a1b75529-istiod-ca-cert\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.910608 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.910591 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-envoy\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.910896 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.910876 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-podinfo\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.916459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.916428 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-token\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:43.916618 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:43.916602 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpp2n\" (UniqueName: \"kubernetes.io/projected/807e1e44-e04f-4ee5-93ac-8450a1b75529-kube-api-access-gpp2n\") pod \"e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:44.069034 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:44.068945 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:44.074909 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:44.074712 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:44.169818 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:44.169787 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:44.356903 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:44.355652 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js"] Sep 15 07:36:44.360650 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:36:44.360610 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807e1e44_e04f_4ee5_93ac_8450a1b75529.slice/crio-3b3a8a28996fe223b69870acb0c96be01e6c47efd079a7979b4c77f24cc7bf57 WatchSource:0}: Error finding container 3b3a8a28996fe223b69870acb0c96be01e6c47efd079a7979b4c77f24cc7bf57: Status 404 returned error can't find the container with id 3b3a8a28996fe223b69870acb0c96be01e6c47efd079a7979b4c77f24cc7bf57 Sep 15 07:36:44.363473 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:44.363433 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:36:44.363576 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:44.363507 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:36:44.363576 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:44.363537 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:36:45.042503 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:45.042448 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:45.045617 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:45.045590 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" event={"ID":"807e1e44-e04f-4ee5-93ac-8450a1b75529","Type":"ContainerStarted","Data":"694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee"} Sep 15 07:36:45.045761 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:45.045637 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" event={"ID":"807e1e44-e04f-4ee5-93ac-8450a1b75529","Type":"ContainerStarted","Data":"3b3a8a28996fe223b69870acb0c96be01e6c47efd079a7979b4c77f24cc7bf57"} Sep 15 07:36:45.069875 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:45.069831 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" podStartSLOduration=2.069814777 podStartE2EDuration="2.069814777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:36:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:36:45.067932006 +0000 UTC m=+1086.049094175" watchObservedRunningTime="2026-09-15 07:36:45.069814777 +0000 UTC m=+1086.050977016" Sep 15 07:36:45.074842 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:45.074814 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:45.079903 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:45.079879 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:45.962318 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:45.962286 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:46.056427 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:46.056394 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:46.057242 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:46.057219 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:36:46.406815 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:46.406785 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:46.503920 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:46.503888 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:46.793327 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:46.793269 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:47.642329 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:47.642290 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:48.353678 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:48.353643 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:48.456886 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:48.456832 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:48.681284 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:48.681203 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:48.852138 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:48.852104 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:48.956093 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:48.956009 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:49.259503 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:49.259472 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:50.152267 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:50.152239 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:50.958314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:50.958283 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:51.055777 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:51.055741 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:52.762274 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:52.762235 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:36:59.078426 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:36:59.078368 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:09.544894 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:09.544822 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:10.186188 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:10.186087 2552 generic.go:356] "Generic (PLEG): container finished" podID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerID="694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee" exitCode=0 Sep 15 07:37:10.186188 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:10.186163 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" event={"ID":"807e1e44-e04f-4ee5-93ac-8450a1b75529","Type":"ContainerDied","Data":"694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee"} Sep 15 07:37:10.186609 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:10.186595 2552 scope.go:117] "RemoveContainer" containerID="694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee" Sep 15 07:37:10.187710 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:10.187681 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:37:10.187798 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:10.187744 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:37:10.187798 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:10.187776 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:37:11.196270 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:11.196235 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" event={"ID":"807e1e44-e04f-4ee5-93ac-8450a1b75529","Type":"ContainerStarted","Data":"737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a"} Sep 15 07:37:11.501766 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:11.501725 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:11.602636 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:11.602599 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:11.962009 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:11.961921 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:12.076426 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.076382 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:37:12.077276 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.077248 2552 patch_prober.go:28] interesting pod/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Sep 15 07:37:12.077485 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.077462 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Sep 15 07:37:12.114334 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.114301 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp"] Sep 15 07:37:12.117900 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.117881 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp" Sep 15 07:37:12.120064 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.120042 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 07:37:12.120170 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.120134 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8jpwk\"" Sep 15 07:37:12.125973 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.125947 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp"] Sep 15 07:37:12.193528 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.193491 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3bc1937-86a5-44fe-9c2a-25bb30e6adc6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b63bda62-plxvp\" (UID: \"d3bc1937-86a5-44fe-9c2a-25bb30e6adc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp" Sep 15 07:37:12.294445 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.294406 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3bc1937-86a5-44fe-9c2a-25bb30e6adc6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b63bda62-plxvp\" (UID: \"d3bc1937-86a5-44fe-9c2a-25bb30e6adc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp" Sep 15 07:37:12.295256 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.295234 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3bc1937-86a5-44fe-9c2a-25bb30e6adc6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b63bda62-plxvp\" (UID: \"d3bc1937-86a5-44fe-9c2a-25bb30e6adc6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp" Sep 15 07:37:12.429992 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.429956 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp" Sep 15 07:37:12.726253 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.726223 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp"] Sep 15 07:37:12.729897 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:37:12.729848 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3bc1937_86a5_44fe_9c2a_25bb30e6adc6.slice/crio-992bd242b3ecdea41e798d039092ade149edd3cb86f9f76adad85753595e0289 WatchSource:0}: Error finding container 992bd242b3ecdea41e798d039092ade149edd3cb86f9f76adad85753595e0289: Status 404 returned error can't find the container with id 992bd242b3ecdea41e798d039092ade149edd3cb86f9f76adad85753595e0289 Sep 15 07:37:12.787118 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:12.787084 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:13.075228 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:13.075195 2552 patch_prober.go:28] interesting pod/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Sep 15 07:37:13.075411 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:13.075280 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Sep 15 07:37:13.214653 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:13.214616 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp" event={"ID":"d3bc1937-86a5-44fe-9c2a-25bb30e6adc6","Type":"ContainerStarted","Data":"1bef0402bc6d84535c2269bec8bb54349d6161f429326e0b96306cd0541c5411"} Sep 15 07:37:13.214818 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:13.214690 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp" event={"ID":"d3bc1937-86a5-44fe-9c2a-25bb30e6adc6","Type":"ContainerStarted","Data":"992bd242b3ecdea41e798d039092ade149edd3cb86f9f76adad85753595e0289"} Sep 15 07:37:13.230468 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:13.230424 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp" podStartSLOduration=1.230409167 podStartE2EDuration="1.230409167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:37:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:37:13.229449776 +0000 UTC m=+1114.210611945" watchObservedRunningTime="2026-09-15 07:37:13.230409167 +0000 UTC m=+1114.211571337" Sep 15 07:37:13.622836 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:13.622796 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:14.075324 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:14.075287 2552 patch_prober.go:28] interesting pod/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Sep 15 07:37:14.075606 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:14.075576 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Sep 15 07:37:14.222058 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:14.222018 2552 generic.go:356] "Generic (PLEG): container finished" podID="d3bc1937-86a5-44fe-9c2a-25bb30e6adc6" containerID="1bef0402bc6d84535c2269bec8bb54349d6161f429326e0b96306cd0541c5411" exitCode=0 Sep 15 07:37:14.222236 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:14.222102 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp" event={"ID":"d3bc1937-86a5-44fe-9c2a-25bb30e6adc6","Type":"ContainerDied","Data":"1bef0402bc6d84535c2269bec8bb54349d6161f429326e0b96306cd0541c5411"} Sep 15 07:37:15.075445 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:15.075408 2552 patch_prober.go:28] interesting pod/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Sep 15 07:37:15.076429 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:15.076025 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Sep 15 07:37:15.393892 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:15.391816 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:15.405636 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:15.405613 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp" Sep 15 07:37:15.426018 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:15.425993 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3bc1937-86a5-44fe-9c2a-25bb30e6adc6-maas-api-service-ca\") pod \"d3bc1937-86a5-44fe-9c2a-25bb30e6adc6\" (UID: \"d3bc1937-86a5-44fe-9c2a-25bb30e6adc6\") " Sep 15 07:37:15.426426 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:15.426398 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3bc1937-86a5-44fe-9c2a-25bb30e6adc6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d3bc1937-86a5-44fe-9c2a-25bb30e6adc6" (UID: "d3bc1937-86a5-44fe-9c2a-25bb30e6adc6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:37:15.493999 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:15.493954 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:15.527118 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:15.527094 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3bc1937-86a5-44fe-9c2a-25bb30e6adc6-maas-api-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:37:15.941415 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:15.941382 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:16.056625 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:16.056586 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:37:16.075602 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:16.075558 2552 patch_prober.go:28] interesting pod/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Sep 15 07:37:16.076095 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:16.075623 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Sep 15 07:37:16.234516 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:16.234492 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp" Sep 15 07:37:16.234684 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:16.234488 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp" event={"ID":"d3bc1937-86a5-44fe-9c2a-25bb30e6adc6","Type":"ContainerDied","Data":"992bd242b3ecdea41e798d039092ade149edd3cb86f9f76adad85753595e0289"} Sep 15 07:37:16.234684 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:16.234638 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="992bd242b3ecdea41e798d039092ade149edd3cb86f9f76adad85753595e0289" Sep 15 07:37:16.634861 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:16.634750 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:17.080177 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:17.080143 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:37:17.081159 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:17.081140 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:37:17.555149 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:17.555104 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:18.158166 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:18.158138 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:18.228572 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:18.228546 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:18.294536 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:18.294478 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:18.368228 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:18.368197 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp"] Sep 15 07:37:18.369249 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:18.369217 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:18.370162 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:18.370140 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b63bda62-plxvp"] Sep 15 07:37:18.439097 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:18.439006 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:18.819170 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:18.819139 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:19.474499 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:19.474468 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3bc1937-86a5-44fe-9c2a-25bb30e6adc6" path="/var/lib/kubelet/pods/d3bc1937-86a5-44fe-9c2a-25bb30e6adc6/volumes" Sep 15 07:37:19.555783 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:19.555747 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js"] Sep 15 07:37:19.555979 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:19.555959 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerName="istio-proxy" containerID="cri-o://737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a" gracePeriod=30 Sep 15 07:37:19.592614 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:19.592581 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:19.780497 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:19.780385 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:19.928766 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:19.928732 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:20.213732 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:20.213651 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:20.537708 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:20.537671 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:20.638031 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:20.637982 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:21.368225 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:21.368190 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:22.343885 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:22.343845 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:22.860841 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:22.860792 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:22.961899 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:22.961843 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:23.135526 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:23.135434 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:24.067131 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.067094 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:24.819684 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.819627 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:37:24.916971 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.916935 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-envoy\") pod \"807e1e44-e04f-4ee5-93ac-8450a1b75529\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " Sep 15 07:37:24.917148 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.916987 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-workload-certs\") pod \"807e1e44-e04f-4ee5-93ac-8450a1b75529\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " Sep 15 07:37:24.917148 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917013 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-workload-socket\") pod \"807e1e44-e04f-4ee5-93ac-8450a1b75529\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " Sep 15 07:37:24.917148 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917039 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-data\") pod \"807e1e44-e04f-4ee5-93ac-8450a1b75529\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " Sep 15 07:37:24.917148 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917062 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-token\") pod \"807e1e44-e04f-4ee5-93ac-8450a1b75529\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " Sep 15 07:37:24.917148 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917122 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-credential-socket\") pod \"807e1e44-e04f-4ee5-93ac-8450a1b75529\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " Sep 15 07:37:24.917418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917168 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/807e1e44-e04f-4ee5-93ac-8450a1b75529-istiod-ca-cert\") pod \"807e1e44-e04f-4ee5-93ac-8450a1b75529\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " Sep 15 07:37:24.917418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917226 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-podinfo\") pod \"807e1e44-e04f-4ee5-93ac-8450a1b75529\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " Sep 15 07:37:24.917418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917262 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpp2n\" (UniqueName: \"kubernetes.io/projected/807e1e44-e04f-4ee5-93ac-8450a1b75529-kube-api-access-gpp2n\") pod \"807e1e44-e04f-4ee5-93ac-8450a1b75529\" (UID: \"807e1e44-e04f-4ee5-93ac-8450a1b75529\") " Sep 15 07:37:24.917418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917279 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "807e1e44-e04f-4ee5-93ac-8450a1b75529" (UID: "807e1e44-e04f-4ee5-93ac-8450a1b75529"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:37:24.917418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917326 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "807e1e44-e04f-4ee5-93ac-8450a1b75529" (UID: "807e1e44-e04f-4ee5-93ac-8450a1b75529"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:37:24.917685 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917483 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-data" (OuterVolumeSpecName: "istio-data") pod "807e1e44-e04f-4ee5-93ac-8450a1b75529" (UID: "807e1e44-e04f-4ee5-93ac-8450a1b75529"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:37:24.917685 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917511 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/807e1e44-e04f-4ee5-93ac-8450a1b75529-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "807e1e44-e04f-4ee5-93ac-8450a1b75529" (UID: "807e1e44-e04f-4ee5-93ac-8450a1b75529"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:37:24.917793 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917688 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-workload-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:37:24.917793 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917707 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-workload-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:37:24.917793 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917713 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "807e1e44-e04f-4ee5-93ac-8450a1b75529" (UID: "807e1e44-e04f-4ee5-93ac-8450a1b75529"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:37:24.917793 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917723 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-data\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:37:24.917793 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.917736 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/807e1e44-e04f-4ee5-93ac-8450a1b75529-istiod-ca-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:37:24.919358 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.919322 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "807e1e44-e04f-4ee5-93ac-8450a1b75529" (UID: "807e1e44-e04f-4ee5-93ac-8450a1b75529"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:37:24.919358 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.919322 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "807e1e44-e04f-4ee5-93ac-8450a1b75529" (UID: "807e1e44-e04f-4ee5-93ac-8450a1b75529"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 07:37:24.919690 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.919668 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-token" (OuterVolumeSpecName: "istio-token") pod "807e1e44-e04f-4ee5-93ac-8450a1b75529" (UID: "807e1e44-e04f-4ee5-93ac-8450a1b75529"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:37:24.919763 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.919699 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/807e1e44-e04f-4ee5-93ac-8450a1b75529-kube-api-access-gpp2n" (OuterVolumeSpecName: "kube-api-access-gpp2n") pod "807e1e44-e04f-4ee5-93ac-8450a1b75529" (UID: "807e1e44-e04f-4ee5-93ac-8450a1b75529"). InnerVolumeSpecName "kube-api-access-gpp2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:37:24.935910 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:24.935883 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:25.019310 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.019268 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-envoy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:37:25.019310 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.019305 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-token\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:37:25.019310 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.019316 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/807e1e44-e04f-4ee5-93ac-8450a1b75529-credential-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:37:25.019548 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.019325 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/807e1e44-e04f-4ee5-93ac-8450a1b75529-istio-podinfo\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:37:25.019548 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.019339 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpp2n\" (UniqueName: \"kubernetes.io/projected/807e1e44-e04f-4ee5-93ac-8450a1b75529-kube-api-access-gpp2n\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:37:25.289556 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.289522 2552 generic.go:356] "Generic (PLEG): container finished" podID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerID="737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a" exitCode=0 Sep 15 07:37:25.289967 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.289576 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" Sep 15 07:37:25.289967 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.289608 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" event={"ID":"807e1e44-e04f-4ee5-93ac-8450a1b75529","Type":"ContainerDied","Data":"737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a"} Sep 15 07:37:25.289967 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.289668 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js" event={"ID":"807e1e44-e04f-4ee5-93ac-8450a1b75529","Type":"ContainerDied","Data":"3b3a8a28996fe223b69870acb0c96be01e6c47efd079a7979b4c77f24cc7bf57"} Sep 15 07:37:25.289967 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.289691 2552 scope.go:117] "RemoveContainer" containerID="737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a" Sep 15 07:37:25.299292 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.299277 2552 scope.go:117] "RemoveContainer" containerID="694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee" Sep 15 07:37:25.307207 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.307191 2552 scope.go:117] "RemoveContainer" containerID="737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a" Sep 15 07:37:25.307474 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:37:25.307455 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a\": container with ID starting with 737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a not found: ID does not exist" containerID="737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a" Sep 15 07:37:25.307538 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.307481 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a"} err="failed to get container status \"737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a\": rpc error: code = NotFound desc = could not find container \"737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a\": container with ID starting with 737867c6613a170eaafa3a92a8e4cb58c52133f7b2937a098510c2ad49a0960a not found: ID does not exist" Sep 15 07:37:25.307538 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.307498 2552 scope.go:117] "RemoveContainer" containerID="694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee" Sep 15 07:37:25.307756 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:37:25.307739 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee\": container with ID starting with 694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee not found: ID does not exist" containerID="694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee" Sep 15 07:37:25.307800 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.307761 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee"} err="failed to get container status \"694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee\": rpc error: code = NotFound desc = could not find container \"694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee\": container with ID starting with 694faaddc04ad59693b7d506220b53e1be804c608a40cef34e1bde883d24fcee not found: ID does not exist" Sep 15 07:37:25.313829 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.313810 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js"] Sep 15 07:37:25.319943 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.319924 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b63bda62-openshift-default-5c7857676d-ln4js"] Sep 15 07:37:25.475412 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.475383 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" path="/var/lib/kubelet/pods/807e1e44-e04f-4ee5-93ac-8450a1b75529/volumes" Sep 15 07:37:25.659180 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:25.659096 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:26.356319 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:26.356286 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:27.154298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:27.154264 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:27.858804 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:27.858773 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:28.569734 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:28.569705 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:28.644609 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:28.644581 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:28.814986 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:28.814952 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:29.143349 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.143316 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:29.277152 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.277115 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz"] Sep 15 07:37:29.277571 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.277552 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerName="istio-proxy" Sep 15 07:37:29.277670 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.277573 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerName="istio-proxy" Sep 15 07:37:29.277670 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.277624 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3bc1937-86a5-44fe-9c2a-25bb30e6adc6" containerName="revoke-keys" Sep 15 07:37:29.277670 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.277634 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3bc1937-86a5-44fe-9c2a-25bb30e6adc6" containerName="revoke-keys" Sep 15 07:37:29.277820 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.277727 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3bc1937-86a5-44fe-9c2a-25bb30e6adc6" containerName="revoke-keys" Sep 15 07:37:29.277820 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.277743 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerName="istio-proxy" Sep 15 07:37:29.277820 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.277754 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerName="istio-proxy" Sep 15 07:37:29.282393 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.282374 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" Sep 15 07:37:29.284695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.284669 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-67d9aa69\"" Sep 15 07:37:29.284794 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.284715 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-67d9aa69-dockercfg-4hblc\"" Sep 15 07:37:29.290861 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.290840 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz"] Sep 15 07:37:29.361915 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.361761 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ff478eb-3a94-4395-9d26-b23c60c100fc-plugins-config-volume\") pod \"payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz\" (UID: \"1ff478eb-3a94-4395-9d26-b23c60c100fc\") " pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" Sep 15 07:37:29.362151 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.362030 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8gcl\" (UniqueName: \"kubernetes.io/projected/1ff478eb-3a94-4395-9d26-b23c60c100fc-kube-api-access-c8gcl\") pod \"payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz\" (UID: \"1ff478eb-3a94-4395-9d26-b23c60c100fc\") " pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" Sep 15 07:37:29.463640 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.463543 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ff478eb-3a94-4395-9d26-b23c60c100fc-plugins-config-volume\") pod \"payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz\" (UID: \"1ff478eb-3a94-4395-9d26-b23c60c100fc\") " pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" Sep 15 07:37:29.463640 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.463632 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8gcl\" (UniqueName: \"kubernetes.io/projected/1ff478eb-3a94-4395-9d26-b23c60c100fc-kube-api-access-c8gcl\") pod \"payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz\" (UID: \"1ff478eb-3a94-4395-9d26-b23c60c100fc\") " pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" Sep 15 07:37:29.464313 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.464288 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ff478eb-3a94-4395-9d26-b23c60c100fc-plugins-config-volume\") pod \"payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz\" (UID: \"1ff478eb-3a94-4395-9d26-b23c60c100fc\") " pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" Sep 15 07:37:29.473549 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.473476 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8gcl\" (UniqueName: \"kubernetes.io/projected/1ff478eb-3a94-4395-9d26-b23c60c100fc-kube-api-access-c8gcl\") pod \"payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz\" (UID: \"1ff478eb-3a94-4395-9d26-b23c60c100fc\") " pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" Sep 15 07:37:29.594334 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.594301 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" Sep 15 07:37:29.866132 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:29.866101 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz"] Sep 15 07:37:29.867433 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:37:29.867409 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ff478eb_3a94_4395_9d26_b23c60c100fc.slice/crio-8a7464ea6d29a752f4cc4bf4f2e10199a7606c2bcaa929078da609d6ae9b7753 WatchSource:0}: Error finding container 8a7464ea6d29a752f4cc4bf4f2e10199a7606c2bcaa929078da609d6ae9b7753: Status 404 returned error can't find the container with id 8a7464ea6d29a752f4cc4bf4f2e10199a7606c2bcaa929078da609d6ae9b7753 Sep 15 07:37:30.318208 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:30.318165 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" event={"ID":"1ff478eb-3a94-4395-9d26-b23c60c100fc","Type":"ContainerStarted","Data":"3c2efc7f7a1029d66ad2e1a6c298e8adebca24bb0b5869ae922afe9952b8325b"} Sep 15 07:37:30.318584 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:30.318227 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" event={"ID":"1ff478eb-3a94-4395-9d26-b23c60c100fc","Type":"ContainerStarted","Data":"8a7464ea6d29a752f4cc4bf4f2e10199a7606c2bcaa929078da609d6ae9b7753"} Sep 15 07:37:30.318584 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:30.318252 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" Sep 15 07:37:30.335368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:30.335321 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" podStartSLOduration=1.335308134 podStartE2EDuration="1.335308134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:37:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:37:30.333203655 +0000 UTC m=+1131.314365823" watchObservedRunningTime="2026-09-15 07:37:30.335308134 +0000 UTC m=+1131.316470301" Sep 15 07:37:30.837233 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:30.837197 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:41.325070 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:41.325039 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" Sep 15 07:37:43.799360 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:43.799329 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:44.389312 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.389281 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:44.421803 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.421770 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462"] Sep 15 07:37:44.422084 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.422038 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" podUID="08c58bcf-9c79-49af-ae5c-47ba44b28daf" containerName="payload-processing" containerID="cri-o://d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a" gracePeriod=30 Sep 15 07:37:44.489528 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.489495 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:44.589496 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.589454 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:44.680849 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.680816 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:44.685471 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.685451 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" Sep 15 07:37:44.810996 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.810961 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08c58bcf-9c79-49af-ae5c-47ba44b28daf-plugins-config-volume\") pod \"08c58bcf-9c79-49af-ae5c-47ba44b28daf\" (UID: \"08c58bcf-9c79-49af-ae5c-47ba44b28daf\") " Sep 15 07:37:44.811504 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.811080 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bj6m6\" (UniqueName: \"kubernetes.io/projected/08c58bcf-9c79-49af-ae5c-47ba44b28daf-kube-api-access-bj6m6\") pod \"08c58bcf-9c79-49af-ae5c-47ba44b28daf\" (UID: \"08c58bcf-9c79-49af-ae5c-47ba44b28daf\") " Sep 15 07:37:44.811577 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.811515 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08c58bcf-9c79-49af-ae5c-47ba44b28daf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "08c58bcf-9c79-49af-ae5c-47ba44b28daf" (UID: "08c58bcf-9c79-49af-ae5c-47ba44b28daf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:37:44.811974 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.811949 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08c58bcf-9c79-49af-ae5c-47ba44b28daf-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:37:44.814782 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.814741 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08c58bcf-9c79-49af-ae5c-47ba44b28daf-kube-api-access-bj6m6" (OuterVolumeSpecName: "kube-api-access-bj6m6") pod "08c58bcf-9c79-49af-ae5c-47ba44b28daf" (UID: "08c58bcf-9c79-49af-ae5c-47ba44b28daf"). InnerVolumeSpecName "kube-api-access-bj6m6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:37:44.913652 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.913575 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bj6m6\" (UniqueName: \"kubernetes.io/projected/08c58bcf-9c79-49af-ae5c-47ba44b28daf-kube-api-access-bj6m6\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:37:44.956817 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:44.956778 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:45.127716 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.127682 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:45.396072 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.396033 2552 generic.go:356] "Generic (PLEG): container finished" podID="08c58bcf-9c79-49af-ae5c-47ba44b28daf" containerID="d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a" exitCode=0 Sep 15 07:37:45.396273 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.396077 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" Sep 15 07:37:45.396273 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.396121 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" event={"ID":"08c58bcf-9c79-49af-ae5c-47ba44b28daf","Type":"ContainerDied","Data":"d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a"} Sep 15 07:37:45.396273 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.396187 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" event={"ID":"08c58bcf-9c79-49af-ae5c-47ba44b28daf","Type":"ContainerDied","Data":"92bc98d229bf0c8ffd73321c2ba3e6da82eabb74da0702bfed80d35e6f017285"} Sep 15 07:37:45.396273 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.396221 2552 scope.go:117] "RemoveContainer" containerID="d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a" Sep 15 07:37:45.407219 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.407204 2552 scope.go:117] "RemoveContainer" containerID="d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a" Sep 15 07:37:45.407485 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:37:45.407455 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a\": container with ID starting with d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a not found: ID does not exist" containerID="d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a" Sep 15 07:37:45.407531 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.407492 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a"} err="failed to get container status \"d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a\": rpc error: code = NotFound desc = could not find container \"d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a\": container with ID starting with d29ce826bcc800b956ad15db78319ad6060a88ed35e336c31feec104bcb3b45a not found: ID does not exist" Sep 15 07:37:45.420423 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.420397 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462"] Sep 15 07:37:45.425963 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.425941 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462"] Sep 15 07:37:45.480906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.476916 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08c58bcf-9c79-49af-ae5c-47ba44b28daf" path="/var/lib/kubelet/pods/08c58bcf-9c79-49af-ae5c-47ba44b28daf/volumes" Sep 15 07:37:45.631922 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.631884 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-autores-f33ec8-84f48486c7-bx462" podUID="08c58bcf-9c79-49af-ae5c-47ba44b28daf" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.40:9004: i/o timeout" Sep 15 07:37:45.971991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:45.971951 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:46.740033 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:46.740004 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:47.759796 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:47.759765 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:48.673399 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:48.673358 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:49.350298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:49.350253 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:49.629257 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:49.629173 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:50.033465 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:50.033438 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:50.374182 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:50.374095 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:51.408842 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:51.408807 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:51.887301 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:51.887257 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:52.647160 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:52.647123 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:53.440398 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:53.440365 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:54.259212 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:54.259176 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:54.949579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:54.949550 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:55.671327 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:55.671294 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:56.542315 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:56.542283 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:37:57.268186 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:37:57.268154 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:04.527198 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:04.527158 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:05.383147 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:05.383117 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz"] Sep 15 07:38:05.383625 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:05.383596 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" podUID="1ff478eb-3a94-4395-9d26-b23c60c100fc" containerName="payload-processing" containerID="cri-o://3c2efc7f7a1029d66ad2e1a6c298e8adebca24bb0b5869ae922afe9952b8325b" gracePeriod=30 Sep 15 07:38:05.502823 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:05.502790 2552 generic.go:356] "Generic (PLEG): container finished" podID="1ff478eb-3a94-4395-9d26-b23c60c100fc" containerID="3c2efc7f7a1029d66ad2e1a6c298e8adebca24bb0b5869ae922afe9952b8325b" exitCode=0 Sep 15 07:38:05.502949 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:05.502856 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" event={"ID":"1ff478eb-3a94-4395-9d26-b23c60c100fc","Type":"ContainerDied","Data":"3c2efc7f7a1029d66ad2e1a6c298e8adebca24bb0b5869ae922afe9952b8325b"} Sep 15 07:38:05.642421 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:05.642363 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" Sep 15 07:38:05.718681 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:05.718637 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ff478eb-3a94-4395-9d26-b23c60c100fc-plugins-config-volume\") pod \"1ff478eb-3a94-4395-9d26-b23c60c100fc\" (UID: \"1ff478eb-3a94-4395-9d26-b23c60c100fc\") " Sep 15 07:38:05.718893 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:05.718765 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8gcl\" (UniqueName: \"kubernetes.io/projected/1ff478eb-3a94-4395-9d26-b23c60c100fc-kube-api-access-c8gcl\") pod \"1ff478eb-3a94-4395-9d26-b23c60c100fc\" (UID: \"1ff478eb-3a94-4395-9d26-b23c60c100fc\") " Sep 15 07:38:05.719053 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:05.719029 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ff478eb-3a94-4395-9d26-b23c60c100fc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1ff478eb-3a94-4395-9d26-b23c60c100fc" (UID: "1ff478eb-3a94-4395-9d26-b23c60c100fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:38:05.719215 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:05.719196 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ff478eb-3a94-4395-9d26-b23c60c100fc-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:38:05.720966 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:05.720935 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ff478eb-3a94-4395-9d26-b23c60c100fc-kube-api-access-c8gcl" (OuterVolumeSpecName: "kube-api-access-c8gcl") pod "1ff478eb-3a94-4395-9d26-b23c60c100fc" (UID: "1ff478eb-3a94-4395-9d26-b23c60c100fc"). InnerVolumeSpecName "kube-api-access-c8gcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:38:05.819849 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:05.819815 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8gcl\" (UniqueName: \"kubernetes.io/projected/1ff478eb-3a94-4395-9d26-b23c60c100fc-kube-api-access-c8gcl\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:38:06.509372 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:06.509342 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" Sep 15 07:38:06.509372 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:06.509360 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz" event={"ID":"1ff478eb-3a94-4395-9d26-b23c60c100fc","Type":"ContainerDied","Data":"8a7464ea6d29a752f4cc4bf4f2e10199a7606c2bcaa929078da609d6ae9b7753"} Sep 15 07:38:06.509617 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:06.509414 2552 scope.go:117] "RemoveContainer" containerID="3c2efc7f7a1029d66ad2e1a6c298e8adebca24bb0b5869ae922afe9952b8325b" Sep 15 07:38:06.534434 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:06.534403 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz"] Sep 15 07:38:06.538494 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:06.538471 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-67d9aa69-56454fd585-jddkz"] Sep 15 07:38:07.474843 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:07.474811 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ff478eb-3a94-4395-9d26-b23c60c100fc" path="/var/lib/kubelet/pods/1ff478eb-3a94-4395-9d26-b23c60c100fc/volumes" Sep 15 07:38:08.554017 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:08.553970 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:09.131076 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:09.131036 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:11.484519 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:11.484465 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:11.585626 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:11.585571 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:13.627449 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:13.627409 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:13.728527 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:13.728483 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:13.828133 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:13.828096 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:13.887374 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:13.887265 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:14.271558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:14.271524 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:23.641241 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:23.641209 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:26.764303 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:26.764269 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:26.970485 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:26.970454 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg"] Sep 15 07:38:26.970752 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:26.970720 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" containerID="cri-o://f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed" gracePeriod=30 Sep 15 07:38:27.011766 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:27.011731 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:27.435707 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:27.435676 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:27.738203 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:27.738169 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:27.877133 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:27.877094 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:29.469071 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:29.469031 2552 patch_prober.go:28] interesting pod/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 15 07:38:29.469522 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:29.469108 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 15 07:38:30.152315 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:30.152279 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:31.010234 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:31.010201 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:31.156529 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:31.156493 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:32.231680 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.231625 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:38:32.283651 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.283622 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-podinfo\") pod \"544eb4bf-b2a2-4376-9502-56aec29c9657\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " Sep 15 07:38:32.283787 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.283677 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hs52\" (UniqueName: \"kubernetes.io/projected/544eb4bf-b2a2-4376-9502-56aec29c9657-kube-api-access-2hs52\") pod \"544eb4bf-b2a2-4376-9502-56aec29c9657\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " Sep 15 07:38:32.283787 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.283702 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-workload-socket\") pod \"544eb4bf-b2a2-4376-9502-56aec29c9657\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " Sep 15 07:38:32.283787 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.283718 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-token\") pod \"544eb4bf-b2a2-4376-9502-56aec29c9657\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " Sep 15 07:38:32.283787 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.283740 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-data\") pod \"544eb4bf-b2a2-4376-9502-56aec29c9657\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " Sep 15 07:38:32.283787 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.283777 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-envoy\") pod \"544eb4bf-b2a2-4376-9502-56aec29c9657\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " Sep 15 07:38:32.284096 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.283797 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544eb4bf-b2a2-4376-9502-56aec29c9657-istiod-ca-cert\") pod \"544eb4bf-b2a2-4376-9502-56aec29c9657\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " Sep 15 07:38:32.284096 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.283837 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-workload-certs\") pod \"544eb4bf-b2a2-4376-9502-56aec29c9657\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " Sep 15 07:38:32.284096 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.283911 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-credential-socket\") pod \"544eb4bf-b2a2-4376-9502-56aec29c9657\" (UID: \"544eb4bf-b2a2-4376-9502-56aec29c9657\") " Sep 15 07:38:32.284096 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.284051 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "544eb4bf-b2a2-4376-9502-56aec29c9657" (UID: "544eb4bf-b2a2-4376-9502-56aec29c9657"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:38:32.284308 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.284251 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "544eb4bf-b2a2-4376-9502-56aec29c9657" (UID: "544eb4bf-b2a2-4376-9502-56aec29c9657"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:38:32.284308 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.284295 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-workload-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:38:32.284454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.284430 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "544eb4bf-b2a2-4376-9502-56aec29c9657" (UID: "544eb4bf-b2a2-4376-9502-56aec29c9657"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:38:32.284663 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.284610 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/544eb4bf-b2a2-4376-9502-56aec29c9657-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "544eb4bf-b2a2-4376-9502-56aec29c9657" (UID: "544eb4bf-b2a2-4376-9502-56aec29c9657"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:38:32.284731 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.284689 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-data" (OuterVolumeSpecName: "istio-data") pod "544eb4bf-b2a2-4376-9502-56aec29c9657" (UID: "544eb4bf-b2a2-4376-9502-56aec29c9657"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:38:32.286130 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.286108 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/544eb4bf-b2a2-4376-9502-56aec29c9657-kube-api-access-2hs52" (OuterVolumeSpecName: "kube-api-access-2hs52") pod "544eb4bf-b2a2-4376-9502-56aec29c9657" (UID: "544eb4bf-b2a2-4376-9502-56aec29c9657"). InnerVolumeSpecName "kube-api-access-2hs52". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:38:32.286462 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.286434 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-token" (OuterVolumeSpecName: "istio-token") pod "544eb4bf-b2a2-4376-9502-56aec29c9657" (UID: "544eb4bf-b2a2-4376-9502-56aec29c9657"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:38:32.286536 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.286488 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "544eb4bf-b2a2-4376-9502-56aec29c9657" (UID: "544eb4bf-b2a2-4376-9502-56aec29c9657"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 07:38:32.286536 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.286494 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "544eb4bf-b2a2-4376-9502-56aec29c9657" (UID: "544eb4bf-b2a2-4376-9502-56aec29c9657"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:38:32.385317 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.385278 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hs52\" (UniqueName: \"kubernetes.io/projected/544eb4bf-b2a2-4376-9502-56aec29c9657-kube-api-access-2hs52\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:38:32.385317 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.385315 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-token\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:38:32.385317 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.385325 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-data\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:38:32.385317 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.385333 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-envoy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:38:32.385558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.385341 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/544eb4bf-b2a2-4376-9502-56aec29c9657-istiod-ca-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:38:32.385558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.385348 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-workload-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:38:32.385558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.385356 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/544eb4bf-b2a2-4376-9502-56aec29c9657-credential-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:38:32.385558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.385365 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/544eb4bf-b2a2-4376-9502-56aec29c9657-istio-podinfo\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:38:32.644854 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.644771 2552 generic.go:356] "Generic (PLEG): container finished" podID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerID="f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed" exitCode=0 Sep 15 07:38:32.644854 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.644814 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" event={"ID":"544eb4bf-b2a2-4376-9502-56aec29c9657","Type":"ContainerDied","Data":"f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed"} Sep 15 07:38:32.645059 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.644832 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" Sep 15 07:38:32.645059 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.644926 2552 scope.go:117] "RemoveContainer" containerID="f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed" Sep 15 07:38:32.645059 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.644912 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg" event={"ID":"544eb4bf-b2a2-4376-9502-56aec29c9657","Type":"ContainerDied","Data":"76ceb3ff0393501a38849a147a973227ba7ab07bedf6deaba0d23d7b6e13cb87"} Sep 15 07:38:32.654734 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.654717 2552 scope.go:117] "RemoveContainer" containerID="efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd" Sep 15 07:38:32.663139 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.663120 2552 scope.go:117] "RemoveContainer" containerID="f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed" Sep 15 07:38:32.663420 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:38:32.663400 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed\": container with ID starting with f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed not found: ID does not exist" containerID="f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed" Sep 15 07:38:32.663468 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.663429 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed"} err="failed to get container status \"f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed\": rpc error: code = NotFound desc = could not find container \"f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed\": container with ID starting with f9d15ba73f63e8076b601ea80d6fb31ce99c37c1aa7321ff243ca970fc90f8ed not found: ID does not exist" Sep 15 07:38:32.663468 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.663446 2552 scope.go:117] "RemoveContainer" containerID="efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd" Sep 15 07:38:32.663706 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:38:32.663686 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd\": container with ID starting with efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd not found: ID does not exist" containerID="efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd" Sep 15 07:38:32.663759 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.663713 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd"} err="failed to get container status \"efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd\": rpc error: code = NotFound desc = could not find container \"efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd\": container with ID starting with efe55fd23f8470690d003f12623531218ac1d2d87fd9fdd40517eff49abc1dcd not found: ID does not exist" Sep 15 07:38:32.674673 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.671615 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg"] Sep 15 07:38:32.676952 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.676930 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-f33ec8-openshift-default-684b546598-5tzpg"] Sep 15 07:38:32.803320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:32.803293 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:33.360284 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:33.360249 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:33.474896 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:33.474832 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" path="/var/lib/kubelet/pods/544eb4bf-b2a2-4376-9502-56aec29c9657/volumes" Sep 15 07:38:34.160324 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:34.160293 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:34.951820 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:34.951780 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:35.066172 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.066136 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:35.179965 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.179934 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:35.483890 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.483842 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:35.593862 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.593827 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c"] Sep 15 07:38:35.594228 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594212 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ff478eb-3a94-4395-9d26-b23c60c100fc" containerName="payload-processing" Sep 15 07:38:35.594228 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594227 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ff478eb-3a94-4395-9d26-b23c60c100fc" containerName="payload-processing" Sep 15 07:38:35.594354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594239 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08c58bcf-9c79-49af-ae5c-47ba44b28daf" containerName="payload-processing" Sep 15 07:38:35.594354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594244 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="08c58bcf-9c79-49af-ae5c-47ba44b28daf" containerName="payload-processing" Sep 15 07:38:35.594354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594267 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" Sep 15 07:38:35.594354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594273 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" Sep 15 07:38:35.594354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594281 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" Sep 15 07:38:35.594354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594286 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" Sep 15 07:38:35.594354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594294 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerName="istio-proxy" Sep 15 07:38:35.594354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594299 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="807e1e44-e04f-4ee5-93ac-8450a1b75529" containerName="istio-proxy" Sep 15 07:38:35.594354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594352 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" Sep 15 07:38:35.594354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594360 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="08c58bcf-9c79-49af-ae5c-47ba44b28daf" containerName="payload-processing" Sep 15 07:38:35.594893 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594369 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="544eb4bf-b2a2-4376-9502-56aec29c9657" containerName="istio-proxy" Sep 15 07:38:35.594893 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.594376 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ff478eb-3a94-4395-9d26-b23c60c100fc" containerName="payload-processing" Sep 15 07:38:35.599043 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.599024 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" Sep 15 07:38:35.601356 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.601336 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-c258f1\"" Sep 15 07:38:35.610400 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.610378 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c"] Sep 15 07:38:35.621596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.621573 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2"] Sep 15 07:38:35.624976 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.624959 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" Sep 15 07:38:35.626750 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.626725 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:35.627444 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.627429 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-c258f1-dockercfg-2ptpj\"" Sep 15 07:38:35.641330 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.641306 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2"] Sep 15 07:38:35.719377 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.718218 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58d2bb7f-37d7-4893-9af0-bf79dbeed272-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c\" (UID: \"58d2bb7f-37d7-4893-9af0-bf79dbeed272\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" Sep 15 07:38:35.719377 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.718316 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97jgd\" (UniqueName: \"kubernetes.io/projected/58d2bb7f-37d7-4893-9af0-bf79dbeed272-kube-api-access-97jgd\") pod \"payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c\" (UID: \"58d2bb7f-37d7-4893-9af0-bf79dbeed272\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" Sep 15 07:38:35.719377 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.718391 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxf2f\" (UniqueName: \"kubernetes.io/projected/40b12be0-6215-40e2-9259-89ea36a028df-kube-api-access-dxf2f\") pod \"payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2\" (UID: \"40b12be0-6215-40e2-9259-89ea36a028df\") " pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" Sep 15 07:38:35.719377 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.718442 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b12be0-6215-40e2-9259-89ea36a028df-plugins-config-volume\") pod \"payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2\" (UID: \"40b12be0-6215-40e2-9259-89ea36a028df\") " pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" Sep 15 07:38:35.819546 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.819452 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58d2bb7f-37d7-4893-9af0-bf79dbeed272-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c\" (UID: \"58d2bb7f-37d7-4893-9af0-bf79dbeed272\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" Sep 15 07:38:35.819546 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.819515 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97jgd\" (UniqueName: \"kubernetes.io/projected/58d2bb7f-37d7-4893-9af0-bf79dbeed272-kube-api-access-97jgd\") pod \"payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c\" (UID: \"58d2bb7f-37d7-4893-9af0-bf79dbeed272\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" Sep 15 07:38:35.819546 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.819542 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxf2f\" (UniqueName: \"kubernetes.io/projected/40b12be0-6215-40e2-9259-89ea36a028df-kube-api-access-dxf2f\") pod \"payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2\" (UID: \"40b12be0-6215-40e2-9259-89ea36a028df\") " pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" Sep 15 07:38:35.819777 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.819566 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b12be0-6215-40e2-9259-89ea36a028df-plugins-config-volume\") pod \"payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2\" (UID: \"40b12be0-6215-40e2-9259-89ea36a028df\") " pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" Sep 15 07:38:35.820217 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.820196 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58d2bb7f-37d7-4893-9af0-bf79dbeed272-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c\" (UID: \"58d2bb7f-37d7-4893-9af0-bf79dbeed272\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" Sep 15 07:38:35.820309 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.820291 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b12be0-6215-40e2-9259-89ea36a028df-plugins-config-volume\") pod \"payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2\" (UID: \"40b12be0-6215-40e2-9259-89ea36a028df\") " pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" Sep 15 07:38:35.833267 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.833235 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxf2f\" (UniqueName: \"kubernetes.io/projected/40b12be0-6215-40e2-9259-89ea36a028df-kube-api-access-dxf2f\") pod \"payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2\" (UID: \"40b12be0-6215-40e2-9259-89ea36a028df\") " pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" Sep 15 07:38:35.833434 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.833416 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97jgd\" (UniqueName: \"kubernetes.io/projected/58d2bb7f-37d7-4893-9af0-bf79dbeed272-kube-api-access-97jgd\") pod \"payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c\" (UID: \"58d2bb7f-37d7-4893-9af0-bf79dbeed272\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" Sep 15 07:38:35.911128 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.911091 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" Sep 15 07:38:35.934448 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:35.934415 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" Sep 15 07:38:36.189301 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:36.189276 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c"] Sep 15 07:38:36.191317 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:38:36.191293 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d2bb7f_37d7_4893_9af0_bf79dbeed272.slice/crio-5f234f095a17c4f2bfeedef996f67938c9bbceb79f3825496e8d1551ace2e7a0 WatchSource:0}: Error finding container 5f234f095a17c4f2bfeedef996f67938c9bbceb79f3825496e8d1551ace2e7a0: Status 404 returned error can't find the container with id 5f234f095a17c4f2bfeedef996f67938c9bbceb79f3825496e8d1551ace2e7a0 Sep 15 07:38:36.221526 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:36.221501 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2"] Sep 15 07:38:36.223230 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:38:36.223202 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b12be0_6215_40e2_9259_89ea36a028df.slice/crio-94190490fa10d5434061c804f71d3125fe5f5a1e74ba07b7a03d7633076eec9a WatchSource:0}: Error finding container 94190490fa10d5434061c804f71d3125fe5f5a1e74ba07b7a03d7633076eec9a: Status 404 returned error can't find the container with id 94190490fa10d5434061c804f71d3125fe5f5a1e74ba07b7a03d7633076eec9a Sep 15 07:38:36.668383 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:36.668342 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" event={"ID":"58d2bb7f-37d7-4893-9af0-bf79dbeed272","Type":"ContainerStarted","Data":"cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12"} Sep 15 07:38:36.668599 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:36.668412 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" event={"ID":"58d2bb7f-37d7-4893-9af0-bf79dbeed272","Type":"ContainerStarted","Data":"5f234f095a17c4f2bfeedef996f67938c9bbceb79f3825496e8d1551ace2e7a0"} Sep 15 07:38:36.668599 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:36.668444 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" Sep 15 07:38:36.669992 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:36.669967 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" event={"ID":"40b12be0-6215-40e2-9259-89ea36a028df","Type":"ContainerStarted","Data":"96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5"} Sep 15 07:38:36.670107 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:36.670008 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" event={"ID":"40b12be0-6215-40e2-9259-89ea36a028df","Type":"ContainerStarted","Data":"94190490fa10d5434061c804f71d3125fe5f5a1e74ba07b7a03d7633076eec9a"} Sep 15 07:38:36.670107 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:36.670095 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" Sep 15 07:38:36.686508 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:36.686433 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" podStartSLOduration=1.686415601 podStartE2EDuration="1.686415601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:38:36.683509534 +0000 UTC m=+1197.664671704" watchObservedRunningTime="2026-09-15 07:38:36.686415601 +0000 UTC m=+1197.667577771" Sep 15 07:38:36.698415 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:36.698366 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" podStartSLOduration=1.698350319 podStartE2EDuration="1.698350319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:38:36.698213865 +0000 UTC m=+1197.679376034" watchObservedRunningTime="2026-09-15 07:38:36.698350319 +0000 UTC m=+1197.679512486" Sep 15 07:38:40.228846 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:40.228812 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:41.045951 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:41.045927 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:38:41.048473 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:41.048450 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:38:41.052989 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:41.052966 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:38:41.055455 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:41.055435 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:38:46.783462 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:46.783425 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:46.858574 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:46.858540 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:46.941711 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:46.941679 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:47.678048 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:47.678016 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" Sep 15 07:38:47.678226 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:47.678128 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" Sep 15 07:38:54.738337 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:54.738295 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:56.680971 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:56.680934 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:56.759253 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:56.759205 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:58.357029 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:58.356997 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:59.226533 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:59.226499 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:38:59.958122 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:38:59.958086 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:00.659666 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:00.659629 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:00.930572 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:00.930479 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:01.461696 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:01.461667 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:02.343334 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:02.343295 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:02.816744 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:02.816711 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:09.431817 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:09.431774 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:09.617771 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:09.617733 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:09.783410 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:09.783366 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:13.769796 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:13.769757 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:20.155397 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:20.155365 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:20.949419 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:20.949381 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:26.605707 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:26.605673 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:26.715927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:26.715891 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:26.819860 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:26.819820 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:26.920115 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:26.920003 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:27.511141 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:27.511110 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:27.612651 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:27.612615 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:27.709777 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:27.709684 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:28.634384 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:28.634343 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:29.198275 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:29.198241 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:35.234077 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:35.233980 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:35.334818 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:35.334760 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:36.007049 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:36.007010 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:41.503583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:41.503546 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:41.604376 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:41.604336 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:41.719636 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:41.719603 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:42.015992 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.015955 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd"] Sep 15 07:39:42.019503 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.019480 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.021733 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.021710 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-526b4727-openshift-default-dockercfg-f6tlh\"" Sep 15 07:39:42.032610 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.032587 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd"] Sep 15 07:39:42.051092 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.051063 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:42.058632 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.058598 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-workload-socket\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.058767 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.058661 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-token\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.058767 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.058726 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-credential-socket\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.058767 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.058761 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg9b7\" (UniqueName: \"kubernetes.io/projected/4d128041-a6ab-444e-b83d-4a865e2f5322-kube-api-access-sg9b7\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.058958 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.058779 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-envoy\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.058958 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.058823 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-data\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.058958 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.058860 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-workload-certs\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.058958 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.058905 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d128041-a6ab-444e-b83d-4a865e2f5322-istiod-ca-cert\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.059165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.058959 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-podinfo\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.148355 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.148312 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:42.159378 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.159352 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-credential-socket\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.159532 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.159386 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg9b7\" (UniqueName: \"kubernetes.io/projected/4d128041-a6ab-444e-b83d-4a865e2f5322-kube-api-access-sg9b7\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.159532 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.159407 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-envoy\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.159532 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.159428 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-data\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.159532 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.159448 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-workload-certs\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.159532 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.159478 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d128041-a6ab-444e-b83d-4a865e2f5322-istiod-ca-cert\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.159532 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.159515 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-podinfo\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.159879 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.159566 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-workload-socket\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.159879 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.159607 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-token\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.160231 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.160196 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-credential-socket\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.160330 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.160223 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d128041-a6ab-444e-b83d-4a865e2f5322-istiod-ca-cert\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.160383 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.160372 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-data\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.160472 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.160449 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-workload-certs\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.160579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.160561 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-workload-socket\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.162165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.162132 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-envoy\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.162390 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.162372 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-podinfo\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.166755 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.166731 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-token\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.167390 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.167372 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg9b7\" (UniqueName: \"kubernetes.io/projected/4d128041-a6ab-444e-b83d-4a865e2f5322-kube-api-access-sg9b7\") pod \"e2e-mt-526b4727-openshift-default-75854694d4-xx8bd\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.331528 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.331434 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:42.660590 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.660567 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd"] Sep 15 07:39:42.663034 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:39:42.663001 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d128041_a6ab_444e_b83d_4a865e2f5322.slice/crio-2fdac51749d3a0a7778d73ed80e44b4a8e7f882ea9f56d69f5ebf1d89f59d95c WatchSource:0}: Error finding container 2fdac51749d3a0a7778d73ed80e44b4a8e7f882ea9f56d69f5ebf1d89f59d95c: Status 404 returned error can't find the container with id 2fdac51749d3a0a7778d73ed80e44b4a8e7f882ea9f56d69f5ebf1d89f59d95c Sep 15 07:39:42.665343 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.665297 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:39:42.665431 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.665396 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:39:42.665474 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:42.665442 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:39:43.008106 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:43.008069 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" event={"ID":"4d128041-a6ab-444e-b83d-4a865e2f5322","Type":"ContainerStarted","Data":"1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a"} Sep 15 07:39:43.008275 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:43.008165 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" event={"ID":"4d128041-a6ab-444e-b83d-4a865e2f5322","Type":"ContainerStarted","Data":"2fdac51749d3a0a7778d73ed80e44b4a8e7f882ea9f56d69f5ebf1d89f59d95c"} Sep 15 07:39:43.028169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:43.028119 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" podStartSLOduration=2.028105463 podStartE2EDuration="2.028105463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:39:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:39:43.02792965 +0000 UTC m=+1264.009091817" watchObservedRunningTime="2026-09-15 07:39:43.028105463 +0000 UTC m=+1264.009267629" Sep 15 07:39:43.331991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:43.331893 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:43.336963 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:43.336941 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:44.013136 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:44.013106 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:44.013991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:44.013968 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:39:47.440330 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:47.440287 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:48.069687 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:48.069648 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:50.518366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:50.518317 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:50.618395 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:50.618360 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:51.136038 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:51.136000 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:51.847641 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:51.847609 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:51.905619 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:51.905583 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:52.460632 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.460598 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:52.555841 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.555802 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:52.560611 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.560582 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr"] Sep 15 07:39:52.564361 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.564345 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" Sep 15 07:39:52.566746 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.566723 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-526b4727-dockercfg-cjv9g\"" Sep 15 07:39:52.566840 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.566764 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-526b4727\"" Sep 15 07:39:52.574889 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.574855 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr"] Sep 15 07:39:52.640581 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.640540 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:39:52.661924 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.661890 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsr4t\" (UniqueName: \"kubernetes.io/projected/c64e8816-936f-4151-bc44-a5979ea0f5f0-kube-api-access-xsr4t\") pod \"payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr\" (UID: \"c64e8816-936f-4151-bc44-a5979ea0f5f0\") " pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" Sep 15 07:39:52.662188 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.662152 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c64e8816-936f-4151-bc44-a5979ea0f5f0-plugins-config-volume\") pod \"payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr\" (UID: \"c64e8816-936f-4151-bc44-a5979ea0f5f0\") " pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" Sep 15 07:39:52.763336 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.763299 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c64e8816-936f-4151-bc44-a5979ea0f5f0-plugins-config-volume\") pod \"payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr\" (UID: \"c64e8816-936f-4151-bc44-a5979ea0f5f0\") " pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" Sep 15 07:39:52.763512 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.763358 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsr4t\" (UniqueName: \"kubernetes.io/projected/c64e8816-936f-4151-bc44-a5979ea0f5f0-kube-api-access-xsr4t\") pod \"payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr\" (UID: \"c64e8816-936f-4151-bc44-a5979ea0f5f0\") " pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" Sep 15 07:39:52.764003 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.763983 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c64e8816-936f-4151-bc44-a5979ea0f5f0-plugins-config-volume\") pod \"payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr\" (UID: \"c64e8816-936f-4151-bc44-a5979ea0f5f0\") " pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" Sep 15 07:39:52.771358 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.771334 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsr4t\" (UniqueName: \"kubernetes.io/projected/c64e8816-936f-4151-bc44-a5979ea0f5f0-kube-api-access-xsr4t\") pod \"payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr\" (UID: \"c64e8816-936f-4151-bc44-a5979ea0f5f0\") " pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" Sep 15 07:39:52.876101 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:52.876068 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" Sep 15 07:39:53.145691 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:53.145667 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr"] Sep 15 07:39:53.148233 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:39:53.148193 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc64e8816_936f_4151_bc44_a5979ea0f5f0.slice/crio-211cc1c1504684adceb4138c3f3456d10d57e7dd92a2e283672a9067842b684b WatchSource:0}: Error finding container 211cc1c1504684adceb4138c3f3456d10d57e7dd92a2e283672a9067842b684b: Status 404 returned error can't find the container with id 211cc1c1504684adceb4138c3f3456d10d57e7dd92a2e283672a9067842b684b Sep 15 07:39:54.070115 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:54.070075 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" event={"ID":"c64e8816-936f-4151-bc44-a5979ea0f5f0","Type":"ContainerStarted","Data":"eccbd011d902532047ed25aa015a2cb10f3d6bb5029e5f7a1c8d910d49d98a36"} Sep 15 07:39:54.070545 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:54.070232 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" Sep 15 07:39:54.070545 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:54.070276 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" event={"ID":"c64e8816-936f-4151-bc44-a5979ea0f5f0","Type":"ContainerStarted","Data":"211cc1c1504684adceb4138c3f3456d10d57e7dd92a2e283672a9067842b684b"} Sep 15 07:39:54.089885 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:39:54.089817 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" podStartSLOduration=2.089798726 podStartE2EDuration="2.089798726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:39:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:39:54.087216404 +0000 UTC m=+1275.068378575" watchObservedRunningTime="2026-09-15 07:39:54.089798726 +0000 UTC m=+1275.070960895" Sep 15 07:40:05.078056 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:05.078021 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" Sep 15 07:40:13.972267 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:13.972095 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:14.748504 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:14.748469 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:15.055123 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:15.055027 2552 patch_prober.go:28] interesting pod/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 15 07:40:15.055123 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:15.055100 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 15 07:40:15.457398 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:15.457317 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:16.032445 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:16.032406 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:16.271671 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:16.271637 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:17.097861 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:17.097773 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:17.194798 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:17.194713 2552 generic.go:356] "Generic (PLEG): container finished" podID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerID="1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a" exitCode=0 Sep 15 07:40:17.194966 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:17.194790 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" event={"ID":"4d128041-a6ab-444e-b83d-4a865e2f5322","Type":"ContainerDied","Data":"1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a"} Sep 15 07:40:17.195247 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:17.195233 2552 scope.go:117] "RemoveContainer" containerID="1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a" Sep 15 07:40:17.196438 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:17.196410 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:40:17.196534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:17.196477 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:40:17.196572 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:17.196509 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:40:17.935989 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:17.935953 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:18.187697 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:18.187606 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:18.204042 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:18.204009 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" event={"ID":"4d128041-a6ab-444e-b83d-4a865e2f5322","Type":"ContainerStarted","Data":"9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015"} Sep 15 07:40:18.332146 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:18.332113 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:40:18.332316 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:18.332278 2552 patch_prober.go:28] interesting pod/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Sep 15 07:40:18.332361 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:18.332333 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Sep 15 07:40:18.775983 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:18.775931 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:18.874017 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:18.873981 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:19.202148 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:19.202064 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:19.332661 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:19.332627 2552 patch_prober.go:28] interesting pod/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Sep 15 07:40:19.332828 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:19.332675 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Sep 15 07:40:20.331942 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:20.331907 2552 patch_prober.go:28] interesting pod/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Sep 15 07:40:20.332334 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:20.331973 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Sep 15 07:40:21.332435 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:21.332403 2552 patch_prober.go:28] interesting pod/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Sep 15 07:40:21.332826 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:21.332467 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Sep 15 07:40:22.332040 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:22.332004 2552 patch_prober.go:28] interesting pod/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Sep 15 07:40:22.332225 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:22.332077 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Sep 15 07:40:23.332679 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:23.332630 2552 patch_prober.go:28] interesting pod/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Sep 15 07:40:23.333215 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:23.332688 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Sep 15 07:40:24.335552 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:24.335518 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:40:24.335973 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:24.335787 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:40:24.336558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:24.336537 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:40:25.222400 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:25.222352 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:25.322647 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:25.322609 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:26.487802 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:26.487764 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:27.330788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:27.330751 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:28.267984 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:28.267948 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:28.960530 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:28.960492 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:29.850033 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:29.849996 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:30.551219 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:30.551188 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:31.342716 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:31.342682 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:36.118991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.118956 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:36.136510 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.136481 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr"] Sep 15 07:40:36.136943 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.136914 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" podUID="c64e8816-936f-4151-bc44-a5979ea0f5f0" containerName="payload-processing" containerID="cri-o://eccbd011d902532047ed25aa015a2cb10f3d6bb5029e5f7a1c8d910d49d98a36" gracePeriod=30 Sep 15 07:40:36.221416 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.221377 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:36.301987 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.301958 2552 generic.go:356] "Generic (PLEG): container finished" podID="c64e8816-936f-4151-bc44-a5979ea0f5f0" containerID="eccbd011d902532047ed25aa015a2cb10f3d6bb5029e5f7a1c8d910d49d98a36" exitCode=0 Sep 15 07:40:36.302138 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.302043 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" event={"ID":"c64e8816-936f-4151-bc44-a5979ea0f5f0","Type":"ContainerDied","Data":"eccbd011d902532047ed25aa015a2cb10f3d6bb5029e5f7a1c8d910d49d98a36"} Sep 15 07:40:36.395748 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.395725 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" Sep 15 07:40:36.513992 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.513956 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsr4t\" (UniqueName: \"kubernetes.io/projected/c64e8816-936f-4151-bc44-a5979ea0f5f0-kube-api-access-xsr4t\") pod \"c64e8816-936f-4151-bc44-a5979ea0f5f0\" (UID: \"c64e8816-936f-4151-bc44-a5979ea0f5f0\") " Sep 15 07:40:36.514192 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.514026 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c64e8816-936f-4151-bc44-a5979ea0f5f0-plugins-config-volume\") pod \"c64e8816-936f-4151-bc44-a5979ea0f5f0\" (UID: \"c64e8816-936f-4151-bc44-a5979ea0f5f0\") " Sep 15 07:40:36.514409 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.514382 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c64e8816-936f-4151-bc44-a5979ea0f5f0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c64e8816-936f-4151-bc44-a5979ea0f5f0" (UID: "c64e8816-936f-4151-bc44-a5979ea0f5f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:40:36.516084 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.516051 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c64e8816-936f-4151-bc44-a5979ea0f5f0-kube-api-access-xsr4t" (OuterVolumeSpecName: "kube-api-access-xsr4t") pod "c64e8816-936f-4151-bc44-a5979ea0f5f0" (UID: "c64e8816-936f-4151-bc44-a5979ea0f5f0"). InnerVolumeSpecName "kube-api-access-xsr4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:40:36.615542 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.615491 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsr4t\" (UniqueName: \"kubernetes.io/projected/c64e8816-936f-4151-bc44-a5979ea0f5f0-kube-api-access-xsr4t\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:40:36.615542 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.615529 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c64e8816-936f-4151-bc44-a5979ea0f5f0-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:40:36.840403 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:36.840364 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:37.309485 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:37.309450 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" Sep 15 07:40:37.309983 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:37.309449 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr" event={"ID":"c64e8816-936f-4151-bc44-a5979ea0f5f0","Type":"ContainerDied","Data":"211cc1c1504684adceb4138c3f3456d10d57e7dd92a2e283672a9067842b684b"} Sep 15 07:40:37.309983 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:37.309596 2552 scope.go:117] "RemoveContainer" containerID="eccbd011d902532047ed25aa015a2cb10f3d6bb5029e5f7a1c8d910d49d98a36" Sep 15 07:40:37.331460 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:37.331437 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr"] Sep 15 07:40:37.333379 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:37.333358 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-526b4727-7ffb78b4df-h5sqr"] Sep 15 07:40:37.421365 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:37.421336 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:37.475145 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:37.475119 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c64e8816-936f-4151-bc44-a5979ea0f5f0" path="/var/lib/kubelet/pods/c64e8816-936f-4151-bc44-a5979ea0f5f0/volumes" Sep 15 07:40:37.480727 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:37.480699 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:38.343132 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:38.343095 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:39.090402 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:39.090363 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:40.077110 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:40.077071 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:40.743684 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:40.743651 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:40.840686 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:40.840646 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:42.311506 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:42.311472 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:42.489758 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:42.489722 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:42.557118 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:42.557083 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:42.751563 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:42.751531 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd"] Sep 15 07:40:42.751761 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:42.751740 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" containerID="cri-o://9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015" gracePeriod=30 Sep 15 07:40:44.702928 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:44.702895 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:44.802928 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:44.802890 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:44.903623 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:44.903591 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:45.694859 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:45.694817 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:48.012194 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.012137 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:40:48.126892 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.126840 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-credential-socket\") pod \"4d128041-a6ab-444e-b83d-4a865e2f5322\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " Sep 15 07:40:48.127063 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.126946 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-workload-socket\") pod \"4d128041-a6ab-444e-b83d-4a865e2f5322\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " Sep 15 07:40:48.127063 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.126972 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-workload-certs\") pod \"4d128041-a6ab-444e-b83d-4a865e2f5322\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " Sep 15 07:40:48.127063 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.126992 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d128041-a6ab-444e-b83d-4a865e2f5322-istiod-ca-cert\") pod \"4d128041-a6ab-444e-b83d-4a865e2f5322\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " Sep 15 07:40:48.127063 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127016 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-envoy\") pod \"4d128041-a6ab-444e-b83d-4a865e2f5322\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " Sep 15 07:40:48.127063 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127044 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-token\") pod \"4d128041-a6ab-444e-b83d-4a865e2f5322\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " Sep 15 07:40:48.127303 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127096 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-podinfo\") pod \"4d128041-a6ab-444e-b83d-4a865e2f5322\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " Sep 15 07:40:48.127303 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127147 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg9b7\" (UniqueName: \"kubernetes.io/projected/4d128041-a6ab-444e-b83d-4a865e2f5322-kube-api-access-sg9b7\") pod \"4d128041-a6ab-444e-b83d-4a865e2f5322\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " Sep 15 07:40:48.127303 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127151 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4d128041-a6ab-444e-b83d-4a865e2f5322" (UID: "4d128041-a6ab-444e-b83d-4a865e2f5322"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:40:48.127303 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127184 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-data\") pod \"4d128041-a6ab-444e-b83d-4a865e2f5322\" (UID: \"4d128041-a6ab-444e-b83d-4a865e2f5322\") " Sep 15 07:40:48.127514 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127307 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4d128041-a6ab-444e-b83d-4a865e2f5322" (UID: "4d128041-a6ab-444e-b83d-4a865e2f5322"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:40:48.127514 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127327 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4d128041-a6ab-444e-b83d-4a865e2f5322" (UID: "4d128041-a6ab-444e-b83d-4a865e2f5322"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:40:48.127514 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127476 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-data" (OuterVolumeSpecName: "istio-data") pod "4d128041-a6ab-444e-b83d-4a865e2f5322" (UID: "4d128041-a6ab-444e-b83d-4a865e2f5322"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:40:48.127736 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127711 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d128041-a6ab-444e-b83d-4a865e2f5322-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4d128041-a6ab-444e-b83d-4a865e2f5322" (UID: "4d128041-a6ab-444e-b83d-4a865e2f5322"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:40:48.127810 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127781 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-workload-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:40:48.127810 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127795 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-data\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:40:48.127810 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127803 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-credential-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:40:48.128004 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.127811 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-workload-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:40:48.129279 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.129242 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4d128041-a6ab-444e-b83d-4a865e2f5322" (UID: "4d128041-a6ab-444e-b83d-4a865e2f5322"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 07:40:48.129389 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.129333 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4d128041-a6ab-444e-b83d-4a865e2f5322" (UID: "4d128041-a6ab-444e-b83d-4a865e2f5322"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:40:48.129745 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.129719 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-token" (OuterVolumeSpecName: "istio-token") pod "4d128041-a6ab-444e-b83d-4a865e2f5322" (UID: "4d128041-a6ab-444e-b83d-4a865e2f5322"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:40:48.129818 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.129757 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d128041-a6ab-444e-b83d-4a865e2f5322-kube-api-access-sg9b7" (OuterVolumeSpecName: "kube-api-access-sg9b7") pod "4d128041-a6ab-444e-b83d-4a865e2f5322" (UID: "4d128041-a6ab-444e-b83d-4a865e2f5322"). InnerVolumeSpecName "kube-api-access-sg9b7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:40:48.228558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.228524 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-podinfo\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:40:48.228558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.228552 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg9b7\" (UniqueName: \"kubernetes.io/projected/4d128041-a6ab-444e-b83d-4a865e2f5322-kube-api-access-sg9b7\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:40:48.228732 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.228565 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d128041-a6ab-444e-b83d-4a865e2f5322-istiod-ca-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:40:48.228732 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.228578 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-envoy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:40:48.228732 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.228588 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d128041-a6ab-444e-b83d-4a865e2f5322-istio-token\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:40:48.369718 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.369640 2552 generic.go:356] "Generic (PLEG): container finished" podID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerID="9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015" exitCode=0 Sep 15 07:40:48.369718 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.369690 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" Sep 15 07:40:48.369930 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.369684 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" event={"ID":"4d128041-a6ab-444e-b83d-4a865e2f5322","Type":"ContainerDied","Data":"9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015"} Sep 15 07:40:48.369930 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.369810 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd" event={"ID":"4d128041-a6ab-444e-b83d-4a865e2f5322","Type":"ContainerDied","Data":"2fdac51749d3a0a7778d73ed80e44b4a8e7f882ea9f56d69f5ebf1d89f59d95c"} Sep 15 07:40:48.369930 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.369838 2552 scope.go:117] "RemoveContainer" containerID="9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015" Sep 15 07:40:48.380734 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.380718 2552 scope.go:117] "RemoveContainer" containerID="1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a" Sep 15 07:40:48.391442 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.391420 2552 scope.go:117] "RemoveContainer" containerID="9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015" Sep 15 07:40:48.391938 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:40:48.391914 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015\": container with ID starting with 9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015 not found: ID does not exist" containerID="9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015" Sep 15 07:40:48.392032 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.391945 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015"} err="failed to get container status \"9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015\": rpc error: code = NotFound desc = could not find container \"9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015\": container with ID starting with 9cab3c4947fb9c965fe72130f0ca1a14e60c5d5476ab94344eefd5f9955c1015 not found: ID does not exist" Sep 15 07:40:48.392032 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.391962 2552 scope.go:117] "RemoveContainer" containerID="1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a" Sep 15 07:40:48.392250 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:40:48.392227 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a\": container with ID starting with 1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a not found: ID does not exist" containerID="1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a" Sep 15 07:40:48.392315 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.392258 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a"} err="failed to get container status \"1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a\": rpc error: code = NotFound desc = could not find container \"1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a\": container with ID starting with 1caa1cb09ae05b8c0bc52dc1ba6e894e21218ecbf3fe869d59ab3644d42d849a not found: ID does not exist" Sep 15 07:40:48.393481 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.393460 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd"] Sep 15 07:40:48.396231 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.396210 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-526b4727-openshift-default-75854694d4-xx8bd"] Sep 15 07:40:48.708303 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.708213 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq"] Sep 15 07:40:48.708696 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.708678 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" Sep 15 07:40:48.708788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.708698 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" Sep 15 07:40:48.708788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.708726 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" Sep 15 07:40:48.708788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.708735 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" Sep 15 07:40:48.708788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.708746 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c64e8816-936f-4151-bc44-a5979ea0f5f0" containerName="payload-processing" Sep 15 07:40:48.708788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.708755 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c64e8816-936f-4151-bc44-a5979ea0f5f0" containerName="payload-processing" Sep 15 07:40:48.709081 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.708883 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" Sep 15 07:40:48.709081 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.708900 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c64e8816-936f-4151-bc44-a5979ea0f5f0" containerName="payload-processing" Sep 15 07:40:48.711610 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.711588 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq" Sep 15 07:40:48.714390 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.714373 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 07:40:48.714492 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.714436 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8jpwk\"" Sep 15 07:40:48.719638 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.719617 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq"] Sep 15 07:40:48.834349 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.834312 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e2d208-60de-4b7b-9eb1-4a07dbf76401-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c258f1-725qq\" (UID: \"72e2d208-60de-4b7b-9eb1-4a07dbf76401\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq" Sep 15 07:40:48.855849 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.855818 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:48.935611 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.935577 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e2d208-60de-4b7b-9eb1-4a07dbf76401-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c258f1-725qq\" (UID: \"72e2d208-60de-4b7b-9eb1-4a07dbf76401\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq" Sep 15 07:40:48.936223 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:48.936205 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e2d208-60de-4b7b-9eb1-4a07dbf76401-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c258f1-725qq\" (UID: \"72e2d208-60de-4b7b-9eb1-4a07dbf76401\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq" Sep 15 07:40:49.023506 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:49.023471 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq" Sep 15 07:40:49.061463 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:49.061369 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:49.165686 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:49.165649 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:49.265558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:49.265522 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:49.294592 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:49.294569 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq"] Sep 15 07:40:49.296497 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:40:49.296470 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e2d208_60de_4b7b_9eb1_4a07dbf76401.slice/crio-965b5e79fb492670e71e2daf1f393c3ddf60598f8add52bd073339099b19ef12 WatchSource:0}: Error finding container 965b5e79fb492670e71e2daf1f393c3ddf60598f8add52bd073339099b19ef12: Status 404 returned error can't find the container with id 965b5e79fb492670e71e2daf1f393c3ddf60598f8add52bd073339099b19ef12 Sep 15 07:40:49.377535 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:49.377511 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq" event={"ID":"72e2d208-60de-4b7b-9eb1-4a07dbf76401","Type":"ContainerStarted","Data":"965b5e79fb492670e71e2daf1f393c3ddf60598f8add52bd073339099b19ef12"} Sep 15 07:40:49.413419 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:49.413391 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:49.474631 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:49.474606 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" path="/var/lib/kubelet/pods/4d128041-a6ab-444e-b83d-4a865e2f5322/volumes" Sep 15 07:40:49.857042 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:49.856940 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:50.383746 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:50.383709 2552 generic.go:356] "Generic (PLEG): container finished" podID="72e2d208-60de-4b7b-9eb1-4a07dbf76401" containerID="b29141a0c9530bf790b6cfc804549d07cae2c83119f76cf2248c3a928447f6b7" exitCode=0 Sep 15 07:40:50.384139 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:50.383784 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq" event={"ID":"72e2d208-60de-4b7b-9eb1-4a07dbf76401","Type":"ContainerDied","Data":"b29141a0c9530bf790b6cfc804549d07cae2c83119f76cf2248c3a928447f6b7"} Sep 15 07:40:50.713704 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:50.713614 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:51.435807 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:51.435763 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:51.532446 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:51.532417 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq" Sep 15 07:40:51.660258 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:51.660170 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e2d208-60de-4b7b-9eb1-4a07dbf76401-maas-api-service-ca\") pod \"72e2d208-60de-4b7b-9eb1-4a07dbf76401\" (UID: \"72e2d208-60de-4b7b-9eb1-4a07dbf76401\") " Sep 15 07:40:51.660527 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:51.660505 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72e2d208-60de-4b7b-9eb1-4a07dbf76401-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72e2d208-60de-4b7b-9eb1-4a07dbf76401" (UID: "72e2d208-60de-4b7b-9eb1-4a07dbf76401"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:40:51.761618 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:51.761573 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e2d208-60de-4b7b-9eb1-4a07dbf76401-maas-api-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:40:52.151744 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:52.151704 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:52.394478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:52.394444 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq" event={"ID":"72e2d208-60de-4b7b-9eb1-4a07dbf76401","Type":"ContainerDied","Data":"965b5e79fb492670e71e2daf1f393c3ddf60598f8add52bd073339099b19ef12"} Sep 15 07:40:52.394478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:52.394462 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq" Sep 15 07:40:52.394693 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:52.394492 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="965b5e79fb492670e71e2daf1f393c3ddf60598f8add52bd073339099b19ef12" Sep 15 07:40:52.951378 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:52.951341 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:53.715281 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:53.715240 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:54.250487 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:54.250456 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:55.989239 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:55.989201 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:56.086713 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.086668 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:56.329326 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.329244 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g"] Sep 15 07:40:56.329702 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.329686 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72e2d208-60de-4b7b-9eb1-4a07dbf76401" containerName="revoke-keys" Sep 15 07:40:56.329775 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.329705 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="72e2d208-60de-4b7b-9eb1-4a07dbf76401" containerName="revoke-keys" Sep 15 07:40:56.329775 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.329761 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="72e2d208-60de-4b7b-9eb1-4a07dbf76401" containerName="revoke-keys" Sep 15 07:40:56.329775 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.329772 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d128041-a6ab-444e-b83d-4a865e2f5322" containerName="istio-proxy" Sep 15 07:40:56.331973 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.331955 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.334336 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.334317 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-3a4c9f-openshift-default-dockercfg-rqvtd\"" Sep 15 07:40:56.346922 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.346896 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g"] Sep 15 07:40:56.352432 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.352404 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:56.464254 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.464219 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:56.506752 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.506717 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-token\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.506959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.506761 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-workload-socket\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.506959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.506782 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-envoy\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.506959 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.506940 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-workload-certs\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.507140 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.506985 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-podinfo\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.507140 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.507014 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqbjx\" (UniqueName: \"kubernetes.io/projected/d69777a3-ea17-44f1-b145-f2f9de1f1318-kube-api-access-lqbjx\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.507140 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.507082 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-credential-socket\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.507245 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.507159 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-data\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.507245 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.507228 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d69777a3-ea17-44f1-b145-f2f9de1f1318-istiod-ca-cert\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.608675 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.608597 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-workload-certs\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.608675 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.608632 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-podinfo\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.608861 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.608774 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqbjx\" (UniqueName: \"kubernetes.io/projected/d69777a3-ea17-44f1-b145-f2f9de1f1318-kube-api-access-lqbjx\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.608958 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.608926 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-credential-socket\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.609052 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.609032 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-data\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.609142 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.609048 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-workload-certs\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.609210 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.609146 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d69777a3-ea17-44f1-b145-f2f9de1f1318-istiod-ca-cert\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.609268 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.609251 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-credential-socket\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.609324 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.609272 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-token\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.609385 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.609341 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-workload-socket\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.609385 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.609359 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-data\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.609385 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.609372 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-envoy\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.609706 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.609680 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-workload-socket\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.609783 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.609765 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d69777a3-ea17-44f1-b145-f2f9de1f1318-istiod-ca-cert\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.611113 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.611090 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-podinfo\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.611251 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.611234 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-envoy\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.616738 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.616714 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqbjx\" (UniqueName: \"kubernetes.io/projected/d69777a3-ea17-44f1-b145-f2f9de1f1318-kube-api-access-lqbjx\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.616821 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.616719 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-token\") pod \"e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.642478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.642455 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:56.916195 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.916168 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g"] Sep 15 07:40:56.918530 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:40:56.918502 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69777a3_ea17_44f1_b145_f2f9de1f1318.slice/crio-846fa308eac3b67d663b4de26a8b066bd7b47f4cd06c9d606baf74db7bed0557 WatchSource:0}: Error finding container 846fa308eac3b67d663b4de26a8b066bd7b47f4cd06c9d606baf74db7bed0557: Status 404 returned error can't find the container with id 846fa308eac3b67d663b4de26a8b066bd7b47f4cd06c9d606baf74db7bed0557 Sep 15 07:40:56.920728 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.920695 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:40:56.920826 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.920758 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:40:56.920826 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:56.920787 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:40:57.425516 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:57.425476 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" event={"ID":"d69777a3-ea17-44f1-b145-f2f9de1f1318","Type":"ContainerStarted","Data":"723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f"} Sep 15 07:40:57.426006 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:57.425538 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" event={"ID":"d69777a3-ea17-44f1-b145-f2f9de1f1318","Type":"ContainerStarted","Data":"846fa308eac3b67d663b4de26a8b066bd7b47f4cd06c9d606baf74db7bed0557"} Sep 15 07:40:57.449676 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:57.449611 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" podStartSLOduration=1.449591693 podStartE2EDuration="1.449591693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:40:57.448729419 +0000 UTC m=+1338.429891599" watchObservedRunningTime="2026-09-15 07:40:57.449591693 +0000 UTC m=+1338.430753862" Sep 15 07:40:57.643203 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:57.643173 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:57.648337 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:57.648312 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:58.431330 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:58.431297 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:58.432246 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:58.432229 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:40:58.644097 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:58.644067 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:59.331745 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:59.331701 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:59.428345 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:59.428306 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:40:59.531633 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:40:59.531582 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:00.325995 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.325955 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:00.384612 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.384579 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2"] Sep 15 07:41:00.384846 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.384822 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" podUID="40b12be0-6215-40e2-9259-89ea36a028df" containerName="payload-processing" containerID="cri-o://96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5" gracePeriod=30 Sep 15 07:41:00.391441 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.391412 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c"] Sep 15 07:41:00.391665 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.391631 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" podUID="58d2bb7f-37d7-4893-9af0-bf79dbeed272" containerName="payload-pre-processing" containerID="cri-o://cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12" gracePeriod=30 Sep 15 07:41:00.658659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.658513 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq"] Sep 15 07:41:00.661210 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.661185 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c258f1-725qq"] Sep 15 07:41:00.664034 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.664014 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" Sep 15 07:41:00.695415 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.695390 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" Sep 15 07:41:00.745788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.745752 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58d2bb7f-37d7-4893-9af0-bf79dbeed272-plugins-config-volume\") pod \"58d2bb7f-37d7-4893-9af0-bf79dbeed272\" (UID: \"58d2bb7f-37d7-4893-9af0-bf79dbeed272\") " Sep 15 07:41:00.745788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.745791 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b12be0-6215-40e2-9259-89ea36a028df-plugins-config-volume\") pod \"40b12be0-6215-40e2-9259-89ea36a028df\" (UID: \"40b12be0-6215-40e2-9259-89ea36a028df\") " Sep 15 07:41:00.746079 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.745832 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97jgd\" (UniqueName: \"kubernetes.io/projected/58d2bb7f-37d7-4893-9af0-bf79dbeed272-kube-api-access-97jgd\") pod \"58d2bb7f-37d7-4893-9af0-bf79dbeed272\" (UID: \"58d2bb7f-37d7-4893-9af0-bf79dbeed272\") " Sep 15 07:41:00.746079 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.745930 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxf2f\" (UniqueName: \"kubernetes.io/projected/40b12be0-6215-40e2-9259-89ea36a028df-kube-api-access-dxf2f\") pod \"40b12be0-6215-40e2-9259-89ea36a028df\" (UID: \"40b12be0-6215-40e2-9259-89ea36a028df\") " Sep 15 07:41:00.746199 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.746132 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58d2bb7f-37d7-4893-9af0-bf79dbeed272-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "58d2bb7f-37d7-4893-9af0-bf79dbeed272" (UID: "58d2bb7f-37d7-4893-9af0-bf79dbeed272"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:41:00.746259 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.746217 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58d2bb7f-37d7-4893-9af0-bf79dbeed272-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:41:00.746309 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.746246 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40b12be0-6215-40e2-9259-89ea36a028df-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40b12be0-6215-40e2-9259-89ea36a028df" (UID: "40b12be0-6215-40e2-9259-89ea36a028df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:41:00.748048 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.748024 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40b12be0-6215-40e2-9259-89ea36a028df-kube-api-access-dxf2f" (OuterVolumeSpecName: "kube-api-access-dxf2f") pod "40b12be0-6215-40e2-9259-89ea36a028df" (UID: "40b12be0-6215-40e2-9259-89ea36a028df"). InnerVolumeSpecName "kube-api-access-dxf2f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:41:00.748156 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.748138 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d2bb7f-37d7-4893-9af0-bf79dbeed272-kube-api-access-97jgd" (OuterVolumeSpecName: "kube-api-access-97jgd") pod "58d2bb7f-37d7-4893-9af0-bf79dbeed272" (UID: "58d2bb7f-37d7-4893-9af0-bf79dbeed272"). InnerVolumeSpecName "kube-api-access-97jgd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:41:00.847129 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.847099 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b12be0-6215-40e2-9259-89ea36a028df-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:41:00.847129 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.847122 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97jgd\" (UniqueName: \"kubernetes.io/projected/58d2bb7f-37d7-4893-9af0-bf79dbeed272-kube-api-access-97jgd\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:41:00.847129 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:00.847132 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxf2f\" (UniqueName: \"kubernetes.io/projected/40b12be0-6215-40e2-9259-89ea36a028df-kube-api-access-dxf2f\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:41:01.100336 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.100303 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:01.250597 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.250563 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:01.450390 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.450362 2552 generic.go:356] "Generic (PLEG): container finished" podID="40b12be0-6215-40e2-9259-89ea36a028df" containerID="96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5" exitCode=0 Sep 15 07:41:01.450546 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.450408 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" Sep 15 07:41:01.450546 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.450440 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" event={"ID":"40b12be0-6215-40e2-9259-89ea36a028df","Type":"ContainerDied","Data":"96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5"} Sep 15 07:41:01.450546 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.450527 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2" event={"ID":"40b12be0-6215-40e2-9259-89ea36a028df","Type":"ContainerDied","Data":"94190490fa10d5434061c804f71d3125fe5f5a1e74ba07b7a03d7633076eec9a"} Sep 15 07:41:01.450695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.450554 2552 scope.go:117] "RemoveContainer" containerID="96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5" Sep 15 07:41:01.451887 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.451743 2552 generic.go:356] "Generic (PLEG): container finished" podID="58d2bb7f-37d7-4893-9af0-bf79dbeed272" containerID="cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12" exitCode=0 Sep 15 07:41:01.451887 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.451779 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" event={"ID":"58d2bb7f-37d7-4893-9af0-bf79dbeed272","Type":"ContainerDied","Data":"cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12"} Sep 15 07:41:01.451887 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.451831 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" event={"ID":"58d2bb7f-37d7-4893-9af0-bf79dbeed272","Type":"ContainerDied","Data":"5f234f095a17c4f2bfeedef996f67938c9bbceb79f3825496e8d1551ace2e7a0"} Sep 15 07:41:01.451887 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.451852 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c" Sep 15 07:41:01.460651 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.460594 2552 scope.go:117] "RemoveContainer" containerID="96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5" Sep 15 07:41:01.460960 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:41:01.460936 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5\": container with ID starting with 96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5 not found: ID does not exist" containerID="96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5" Sep 15 07:41:01.461042 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.460965 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5"} err="failed to get container status \"96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5\": rpc error: code = NotFound desc = could not find container \"96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5\": container with ID starting with 96ccda71af3a193156e40eb640429ebd611e5fcf48039e5ade694aeb81e815d5 not found: ID does not exist" Sep 15 07:41:01.461042 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.460982 2552 scope.go:117] "RemoveContainer" containerID="cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12" Sep 15 07:41:01.469007 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.468989 2552 scope.go:117] "RemoveContainer" containerID="cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12" Sep 15 07:41:01.469285 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:41:01.469264 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12\": container with ID starting with cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12 not found: ID does not exist" containerID="cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12" Sep 15 07:41:01.469362 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.469294 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12"} err="failed to get container status \"cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12\": rpc error: code = NotFound desc = could not find container \"cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12\": container with ID starting with cee940675eeb32cfd06761533ed6dd5c82d6db77b789bedf3a758418d0885d12 not found: ID does not exist" Sep 15 07:41:01.475112 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.475092 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72e2d208-60de-4b7b-9eb1-4a07dbf76401" path="/var/lib/kubelet/pods/72e2d208-60de-4b7b-9eb1-4a07dbf76401/volumes" Sep 15 07:41:01.475355 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.475341 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2"] Sep 15 07:41:01.479152 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.479132 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-c258f1-54bb895db5-95sv2"] Sep 15 07:41:01.488157 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.488137 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c"] Sep 15 07:41:01.495663 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.495642 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c258f1-764cdcbb66-t568c"] Sep 15 07:41:01.545037 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:01.545013 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:02.370797 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:02.370702 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:03.249571 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.249538 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:03.312785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.312746 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:03.475396 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.475364 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40b12be0-6215-40e2-9259-89ea36a028df" path="/var/lib/kubelet/pods/40b12be0-6215-40e2-9259-89ea36a028df/volumes" Sep 15 07:41:03.475777 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.475710 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58d2bb7f-37d7-4893-9af0-bf79dbeed272" path="/var/lib/kubelet/pods/58d2bb7f-37d7-4893-9af0-bf79dbeed272/volumes" Sep 15 07:41:03.637803 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.637714 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:03.737413 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.737381 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:03.842560 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.842520 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh"] Sep 15 07:41:03.843123 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.843089 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40b12be0-6215-40e2-9259-89ea36a028df" containerName="payload-processing" Sep 15 07:41:03.843123 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.843115 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="40b12be0-6215-40e2-9259-89ea36a028df" containerName="payload-processing" Sep 15 07:41:03.843336 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.843163 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d2bb7f-37d7-4893-9af0-bf79dbeed272" containerName="payload-pre-processing" Sep 15 07:41:03.843336 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.843173 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d2bb7f-37d7-4893-9af0-bf79dbeed272" containerName="payload-pre-processing" Sep 15 07:41:03.843336 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.843268 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="40b12be0-6215-40e2-9259-89ea36a028df" containerName="payload-processing" Sep 15 07:41:03.843336 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.843291 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d2bb7f-37d7-4893-9af0-bf79dbeed272" containerName="payload-pre-processing" Sep 15 07:41:03.846148 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.846131 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" Sep 15 07:41:03.848554 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.848536 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-3a4c9f\"" Sep 15 07:41:03.848631 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.848582 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-3a4c9f-dockercfg-zvzjz\"" Sep 15 07:41:03.852920 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.852890 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk"] Sep 15 07:41:03.855758 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.855733 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" Sep 15 07:41:03.857693 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.857673 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh"] Sep 15 07:41:03.874555 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.873640 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk"] Sep 15 07:41:03.875308 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.875281 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3355ae2f-9b9b-46ef-8212-651aa8884535-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh\" (UID: \"3355ae2f-9b9b-46ef-8212-651aa8884535\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" Sep 15 07:41:03.875469 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.875445 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8506cc5b-22f3-48b0-be74-de711496d948-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk\" (UID: \"8506cc5b-22f3-48b0-be74-de711496d948\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" Sep 15 07:41:03.875652 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.875637 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56dlc\" (UniqueName: \"kubernetes.io/projected/3355ae2f-9b9b-46ef-8212-651aa8884535-kube-api-access-56dlc\") pod \"payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh\" (UID: \"3355ae2f-9b9b-46ef-8212-651aa8884535\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" Sep 15 07:41:03.875790 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.875775 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm462\" (UniqueName: \"kubernetes.io/projected/8506cc5b-22f3-48b0-be74-de711496d948-kube-api-access-sm462\") pod \"payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk\" (UID: \"8506cc5b-22f3-48b0-be74-de711496d948\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" Sep 15 07:41:03.981912 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.980085 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8506cc5b-22f3-48b0-be74-de711496d948-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk\" (UID: \"8506cc5b-22f3-48b0-be74-de711496d948\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" Sep 15 07:41:03.981912 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.980157 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56dlc\" (UniqueName: \"kubernetes.io/projected/3355ae2f-9b9b-46ef-8212-651aa8884535-kube-api-access-56dlc\") pod \"payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh\" (UID: \"3355ae2f-9b9b-46ef-8212-651aa8884535\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" Sep 15 07:41:03.981912 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.980200 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm462\" (UniqueName: \"kubernetes.io/projected/8506cc5b-22f3-48b0-be74-de711496d948-kube-api-access-sm462\") pod \"payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk\" (UID: \"8506cc5b-22f3-48b0-be74-de711496d948\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" Sep 15 07:41:03.981912 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.980239 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3355ae2f-9b9b-46ef-8212-651aa8884535-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh\" (UID: \"3355ae2f-9b9b-46ef-8212-651aa8884535\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" Sep 15 07:41:03.981912 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.981145 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3355ae2f-9b9b-46ef-8212-651aa8884535-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh\" (UID: \"3355ae2f-9b9b-46ef-8212-651aa8884535\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" Sep 15 07:41:03.981912 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.981809 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8506cc5b-22f3-48b0-be74-de711496d948-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk\" (UID: \"8506cc5b-22f3-48b0-be74-de711496d948\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" Sep 15 07:41:03.990734 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.990701 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm462\" (UniqueName: \"kubernetes.io/projected/8506cc5b-22f3-48b0-be74-de711496d948-kube-api-access-sm462\") pod \"payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk\" (UID: \"8506cc5b-22f3-48b0-be74-de711496d948\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" Sep 15 07:41:03.990903 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:03.990812 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56dlc\" (UniqueName: \"kubernetes.io/projected/3355ae2f-9b9b-46ef-8212-651aa8884535-kube-api-access-56dlc\") pod \"payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh\" (UID: \"3355ae2f-9b9b-46ef-8212-651aa8884535\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" Sep 15 07:41:04.010920 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:04.010895 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:04.157280 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:04.157244 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" Sep 15 07:41:04.166940 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:04.166912 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" Sep 15 07:41:04.170413 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:04.170384 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:04.436441 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:04.436415 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh"] Sep 15 07:41:04.438901 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:41:04.438857 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3355ae2f_9b9b_46ef_8212_651aa8884535.slice/crio-d7cefe2d11ba888dcc9547be6bd41f1d679f7a7f04060b6e5f6243fead3212b4 WatchSource:0}: Error finding container d7cefe2d11ba888dcc9547be6bd41f1d679f7a7f04060b6e5f6243fead3212b4: Status 404 returned error can't find the container with id d7cefe2d11ba888dcc9547be6bd41f1d679f7a7f04060b6e5f6243fead3212b4 Sep 15 07:41:04.450643 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:04.450620 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk"] Sep 15 07:41:04.452149 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:41:04.452126 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8506cc5b_22f3_48b0_be74_de711496d948.slice/crio-18c20085e7baca73a7275c08818ab51317bb2bfeac9eaaee54b3ac583e27cd54 WatchSource:0}: Error finding container 18c20085e7baca73a7275c08818ab51317bb2bfeac9eaaee54b3ac583e27cd54: Status 404 returned error can't find the container with id 18c20085e7baca73a7275c08818ab51317bb2bfeac9eaaee54b3ac583e27cd54 Sep 15 07:41:04.471511 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:04.471486 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" event={"ID":"8506cc5b-22f3-48b0-be74-de711496d948","Type":"ContainerStarted","Data":"18c20085e7baca73a7275c08818ab51317bb2bfeac9eaaee54b3ac583e27cd54"} Sep 15 07:41:04.473087 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:04.473053 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" event={"ID":"3355ae2f-9b9b-46ef-8212-651aa8884535","Type":"ContainerStarted","Data":"d7cefe2d11ba888dcc9547be6bd41f1d679f7a7f04060b6e5f6243fead3212b4"} Sep 15 07:41:05.479619 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:05.479589 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" event={"ID":"8506cc5b-22f3-48b0-be74-de711496d948","Type":"ContainerStarted","Data":"86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f"} Sep 15 07:41:05.480073 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:05.479639 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" Sep 15 07:41:05.481273 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:05.481244 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" event={"ID":"3355ae2f-9b9b-46ef-8212-651aa8884535","Type":"ContainerStarted","Data":"0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777"} Sep 15 07:41:05.481367 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:05.481311 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" Sep 15 07:41:05.497113 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:05.497071 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" podStartSLOduration=2.497055887 podStartE2EDuration="2.497055887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:41:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:41:05.494919103 +0000 UTC m=+1346.476081270" watchObservedRunningTime="2026-09-15 07:41:05.497055887 +0000 UTC m=+1346.478218053" Sep 15 07:41:05.511745 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:05.511700 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" podStartSLOduration=2.5116844990000002 podStartE2EDuration="2.511684499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:41:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:41:05.510466063 +0000 UTC m=+1346.491628233" watchObservedRunningTime="2026-09-15 07:41:05.511684499 +0000 UTC m=+1346.492846666" Sep 15 07:41:05.720541 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:05.720513 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:05.825214 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:05.825132 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:06.065462 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:06.065428 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:06.442598 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:06.442567 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:11.904041 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:11.904001 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:12.135349 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.135315 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:12.843036 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.842993 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk"] Sep 15 07:41:12.846302 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.846280 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:12.848638 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.848616 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-c3abd1-openshift-default-dockercfg-l5mq7\"" Sep 15 07:41:12.858965 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.858942 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk"] Sep 15 07:41:12.865062 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.865035 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:12.962423 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.961927 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:12.974049 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.974018 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istiod-ca-cert\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:12.974259 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.974081 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-workload-certs\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:12.974259 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.974115 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vxt6\" (UniqueName: \"kubernetes.io/projected/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-kube-api-access-7vxt6\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:12.974259 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.974154 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-data\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:12.974259 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.974222 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-workload-socket\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:12.974259 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.974250 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-token\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:12.974498 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.974294 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-credential-socket\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:12.974498 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.974404 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-podinfo\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:12.974498 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:12.974462 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-envoy\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076172 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076137 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istiod-ca-cert\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076376 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076185 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-workload-certs\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076376 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076210 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vxt6\" (UniqueName: \"kubernetes.io/projected/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-kube-api-access-7vxt6\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076376 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076238 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-data\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076376 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076272 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-workload-socket\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076376 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076286 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-token\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076376 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076309 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-credential-socket\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076693 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076382 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-podinfo\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076693 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076424 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-envoy\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076693 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076639 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-workload-socket\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076846 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076692 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-workload-certs\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076846 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076761 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-credential-socket\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076893 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-data\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.076991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.076972 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istiod-ca-cert\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.078739 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.078718 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-envoy\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.079015 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.078996 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-podinfo\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.090422 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.090402 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-token\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.092033 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.092017 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vxt6\" (UniqueName: \"kubernetes.io/projected/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-kube-api-access-7vxt6\") pod \"e2e-noten-c3abd1-openshift-default-585df76644-lzcvk\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.161347 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.161272 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:13.437995 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.437973 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk"] Sep 15 07:41:13.439542 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:41:13.439509 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04634f3b_130b_4ad0_8bee_78f5ce8a6e53.slice/crio-d109a9a276a77822ed7a73d0838d532f7e92862d78e2de7390b20dde0d2d8d48 WatchSource:0}: Error finding container d109a9a276a77822ed7a73d0838d532f7e92862d78e2de7390b20dde0d2d8d48: Status 404 returned error can't find the container with id d109a9a276a77822ed7a73d0838d532f7e92862d78e2de7390b20dde0d2d8d48 Sep 15 07:41:13.441943 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.441914 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:41:13.442010 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.441978 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:41:13.442010 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.442006 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:41:13.526899 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.526852 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" event={"ID":"04634f3b-130b-4ad0-8bee-78f5ce8a6e53","Type":"ContainerStarted","Data":"d02f78dc4a2b85408fa124a42e146fb9969eaec0d6670842eae6a832ef846e82"} Sep 15 07:41:13.527023 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.526921 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" event={"ID":"04634f3b-130b-4ad0-8bee-78f5ce8a6e53","Type":"ContainerStarted","Data":"d109a9a276a77822ed7a73d0838d532f7e92862d78e2de7390b20dde0d2d8d48"} Sep 15 07:41:13.548218 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.548169 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" podStartSLOduration=1.5481467119999999 podStartE2EDuration="1.548146712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:41:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:41:13.544330846 +0000 UTC m=+1354.525493014" watchObservedRunningTime="2026-09-15 07:41:13.548146712 +0000 UTC m=+1354.529308879" Sep 15 07:41:13.622484 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.622445 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:13.833755 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.833714 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:13.934594 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:13.934563 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:14.040500 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:14.040472 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:14.161520 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:14.161437 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:14.167131 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:14.167109 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:14.532813 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:14.532766 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:14.533705 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:14.533682 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:41:16.489616 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:16.489589 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" Sep 15 07:41:16.490002 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:16.489649 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" Sep 15 07:41:25.384269 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:25.384234 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:26.077676 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:26.077639 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:26.960928 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:26.960894 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:27.736094 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:27.736054 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:28.449375 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:28.449338 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:29.059157 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:29.058379 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": read tcp 10.134.0.2:59372->10.134.0.53:15021: read: connection reset by peer" start-of-body= Sep 15 07:41:29.059157 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:29.058463 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": read tcp 10.134.0.2:59372->10.134.0.53:15021: read: connection reset by peer" Sep 15 07:41:29.254335 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:29.254301 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:29.355105 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:29.355029 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:29.626060 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:29.625978 2552 generic.go:356] "Generic (PLEG): container finished" podID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerID="723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f" exitCode=0 Sep 15 07:41:29.626426 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:29.626055 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" event={"ID":"d69777a3-ea17-44f1-b145-f2f9de1f1318","Type":"ContainerDied","Data":"723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f"} Sep 15 07:41:29.626523 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:29.626506 2552 scope.go:117] "RemoveContainer" containerID="723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f" Sep 15 07:41:29.627682 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:29.627653 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:41:29.627753 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:29.627721 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:41:29.627753 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:29.627750 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:41:29.660241 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:29.660217 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:29.982367 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:29.982283 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:30.401443 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:30.401331 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:30.633370 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:30.633339 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" event={"ID":"d69777a3-ea17-44f1-b145-f2f9de1f1318","Type":"ContainerStarted","Data":"e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780"} Sep 15 07:41:30.643052 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:30.643022 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:41:30.643248 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:30.643227 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Sep 15 07:41:30.643340 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:30.643300 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Sep 15 07:41:31.643231 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:31.643182 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Sep 15 07:41:31.643647 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:31.643244 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Sep 15 07:41:32.251073 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:32.251027 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:32.643312 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:32.643217 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Sep 15 07:41:32.643312 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:32.643278 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Sep 15 07:41:32.948261 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:32.948171 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:33.047726 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:33.047687 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:33.643462 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:33.643431 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Sep 15 07:41:33.643943 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:33.643490 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Sep 15 07:41:33.794619 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:33.794581 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:34.211660 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:34.211623 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:34.643243 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:34.643204 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Sep 15 07:41:34.643425 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:34.643279 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Sep 15 07:41:34.859319 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:34.859283 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:35.605154 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:35.605125 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:35.642998 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:35.642965 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Sep 15 07:41:35.643153 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:35.643026 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Sep 15 07:41:36.339981 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:36.339948 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:36.646376 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:36.646298 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:41:36.646590 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:36.646569 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:41:36.647353 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:36.647329 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:41:37.162628 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:37.162599 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:37.324444 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:37.324336 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:37.424840 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:37.424798 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:37.649691 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:37.649595 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:38.051944 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:38.051910 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:41:59.377359 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:41:59.377327 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:00.364395 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:00.364359 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:01.087100 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:01.087065 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:01.760197 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:01.760159 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:03.479446 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:03.479407 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:03.580376 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:03.580337 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:04.209504 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:04.209467 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:04.962719 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:04.962690 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:11.160022 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:11.159986 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:13.288675 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:13.288642 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:13.390129 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:13.390088 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:17.445880 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:17.445846 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:17.547883 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:17.547842 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:18.000268 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:18.000230 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:20.634806 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:20.634778 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:20.693660 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:20.693629 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:20.937045 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:20.936958 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:21.003279 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.003232 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:21.237008 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.236979 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:21.431629 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.431596 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:21.549346 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.549266 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx"] Sep 15 07:42:21.552905 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.552889 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" Sep 15 07:42:21.555348 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.555321 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b21d1ff2-dockercfg-sbwts\"" Sep 15 07:42:21.555469 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.555416 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b21d1ff2\"" Sep 15 07:42:21.565709 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.565685 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx"] Sep 15 07:42:21.567968 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.567942 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn"] Sep 15 07:42:21.571788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.571770 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" Sep 15 07:42:21.578197 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.578174 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn"] Sep 15 07:42:21.678627 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.678579 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86c8f32e-f6d9-4996-aa69-639be20202f3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx\" (UID: \"86c8f32e-f6d9-4996-aa69-639be20202f3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" Sep 15 07:42:21.679145 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.678713 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wcc2\" (UniqueName: \"kubernetes.io/projected/03157279-7fb5-48b9-9fbf-06ee72000524-kube-api-access-9wcc2\") pod \"payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn\" (UID: \"03157279-7fb5-48b9-9fbf-06ee72000524\") " pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" Sep 15 07:42:21.679145 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.678796 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn8bj\" (UniqueName: \"kubernetes.io/projected/86c8f32e-f6d9-4996-aa69-639be20202f3-kube-api-access-pn8bj\") pod \"payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx\" (UID: \"86c8f32e-f6d9-4996-aa69-639be20202f3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" Sep 15 07:42:21.679145 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.678850 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03157279-7fb5-48b9-9fbf-06ee72000524-plugins-config-volume\") pod \"payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn\" (UID: \"03157279-7fb5-48b9-9fbf-06ee72000524\") " pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" Sep 15 07:42:21.779744 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.779708 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wcc2\" (UniqueName: \"kubernetes.io/projected/03157279-7fb5-48b9-9fbf-06ee72000524-kube-api-access-9wcc2\") pod \"payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn\" (UID: \"03157279-7fb5-48b9-9fbf-06ee72000524\") " pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" Sep 15 07:42:21.779918 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.779768 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn8bj\" (UniqueName: \"kubernetes.io/projected/86c8f32e-f6d9-4996-aa69-639be20202f3-kube-api-access-pn8bj\") pod \"payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx\" (UID: \"86c8f32e-f6d9-4996-aa69-639be20202f3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" Sep 15 07:42:21.779918 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.779803 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03157279-7fb5-48b9-9fbf-06ee72000524-plugins-config-volume\") pod \"payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn\" (UID: \"03157279-7fb5-48b9-9fbf-06ee72000524\") " pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" Sep 15 07:42:21.779918 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.779835 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86c8f32e-f6d9-4996-aa69-639be20202f3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx\" (UID: \"86c8f32e-f6d9-4996-aa69-639be20202f3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" Sep 15 07:42:21.780585 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.780555 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03157279-7fb5-48b9-9fbf-06ee72000524-plugins-config-volume\") pod \"payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn\" (UID: \"03157279-7fb5-48b9-9fbf-06ee72000524\") " pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" Sep 15 07:42:21.780585 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.780573 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86c8f32e-f6d9-4996-aa69-639be20202f3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx\" (UID: \"86c8f32e-f6d9-4996-aa69-639be20202f3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" Sep 15 07:42:21.788482 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.788459 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn8bj\" (UniqueName: \"kubernetes.io/projected/86c8f32e-f6d9-4996-aa69-639be20202f3-kube-api-access-pn8bj\") pod \"payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx\" (UID: \"86c8f32e-f6d9-4996-aa69-639be20202f3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" Sep 15 07:42:21.788611 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.788581 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wcc2\" (UniqueName: \"kubernetes.io/projected/03157279-7fb5-48b9-9fbf-06ee72000524-kube-api-access-9wcc2\") pod \"payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn\" (UID: \"03157279-7fb5-48b9-9fbf-06ee72000524\") " pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" Sep 15 07:42:21.864456 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.864379 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" Sep 15 07:42:21.883814 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:21.883792 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" Sep 15 07:42:22.143769 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:22.143744 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx"] Sep 15 07:42:22.145290 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:42:22.145262 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86c8f32e_f6d9_4996_aa69_639be20202f3.slice/crio-41dc86189372aeb8b514307bda791934e5b6a2a8d768ab98f7c0358ca8d5caf7 WatchSource:0}: Error finding container 41dc86189372aeb8b514307bda791934e5b6a2a8d768ab98f7c0358ca8d5caf7: Status 404 returned error can't find the container with id 41dc86189372aeb8b514307bda791934e5b6a2a8d768ab98f7c0358ca8d5caf7 Sep 15 07:42:22.179736 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:22.179714 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn"] Sep 15 07:42:22.181807 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:42:22.181783 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03157279_7fb5_48b9_9fbf_06ee72000524.slice/crio-fd1b7247f47594abd9242ea54fb421584039e06ca2e190ab5e20765d517154e0 WatchSource:0}: Error finding container fd1b7247f47594abd9242ea54fb421584039e06ca2e190ab5e20765d517154e0: Status 404 returned error can't find the container with id fd1b7247f47594abd9242ea54fb421584039e06ca2e190ab5e20765d517154e0 Sep 15 07:42:22.627902 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:22.627841 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:22.921761 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:22.921664 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" event={"ID":"03157279-7fb5-48b9-9fbf-06ee72000524","Type":"ContainerStarted","Data":"2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01"} Sep 15 07:42:22.921761 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:22.921729 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" event={"ID":"03157279-7fb5-48b9-9fbf-06ee72000524","Type":"ContainerStarted","Data":"fd1b7247f47594abd9242ea54fb421584039e06ca2e190ab5e20765d517154e0"} Sep 15 07:42:22.921761 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:22.921758 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" Sep 15 07:42:22.923554 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:22.923527 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" event={"ID":"86c8f32e-f6d9-4996-aa69-639be20202f3","Type":"ContainerStarted","Data":"af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e"} Sep 15 07:42:22.923679 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:22.923580 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" event={"ID":"86c8f32e-f6d9-4996-aa69-639be20202f3","Type":"ContainerStarted","Data":"41dc86189372aeb8b514307bda791934e5b6a2a8d768ab98f7c0358ca8d5caf7"} Sep 15 07:42:22.923679 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:22.923610 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" Sep 15 07:42:22.942240 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:22.942190 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" podStartSLOduration=1.942170663 podStartE2EDuration="1.942170663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:42:22.938449549 +0000 UTC m=+1423.919611716" watchObservedRunningTime="2026-09-15 07:42:22.942170663 +0000 UTC m=+1423.923332832" Sep 15 07:42:22.955110 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:22.955068 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" podStartSLOduration=1.955056902 podStartE2EDuration="1.955056902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:42:22.953670827 +0000 UTC m=+1423.934832995" watchObservedRunningTime="2026-09-15 07:42:22.955056902 +0000 UTC m=+1423.936219068" Sep 15 07:42:25.300120 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:25.300083 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:33.931044 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:33.930953 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" Sep 15 07:42:33.931044 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:33.931033 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" Sep 15 07:42:35.820637 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:35.820605 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:38.325225 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:38.325185 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:38.560614 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:38.560581 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:39.390189 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:39.390151 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:39.551645 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:39.551614 2552 scope.go:117] "RemoveContainer" containerID="e9a06b51d4b9ad9f9af9e4dafd2c3d658cd171fea2f37ab81d54622b0807e581" Sep 15 07:42:39.560172 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:39.560152 2552 scope.go:117] "RemoveContainer" containerID="58d9a2df86dcab772ae4f08f038b309ddd3e00f232090ccca9872c75ed7c8b4b" Sep 15 07:42:40.351547 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:40.351513 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:41.152815 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:41.152785 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:42.040576 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:42.040545 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:42.856605 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:42.856576 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:44.333881 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:44.333835 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:44.564714 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:44.564678 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:46.116338 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:46.116306 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:46.218482 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:46.218449 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:46.319695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:46.319643 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:47.045091 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:47.045060 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:47.839735 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:47.839702 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:48.536585 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:48.536549 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:49.352634 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:49.352598 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:50.167739 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:50.167707 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:50.794890 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:50.794839 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:50.892579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:50.892546 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk"] Sep 15 07:42:50.892832 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:50.892808 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" podUID="04634f3b-130b-4ad0-8bee-78f5ce8a6e53" containerName="istio-proxy" containerID="cri-o://d02f78dc4a2b85408fa124a42e146fb9969eaec0d6670842eae6a832ef846e82" gracePeriod=30 Sep 15 07:42:50.908264 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:50.908235 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:51.266418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:51.266379 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:51.693726 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:51.693634 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx"] Sep 15 07:42:51.693924 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:51.693894 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" podUID="7a9aa094-c37b-4f88-a7fc-4a27355cfe8c" containerName="payload-processing" containerID="cri-o://63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30" gracePeriod=30 Sep 15 07:42:51.703809 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:51.703769 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:51.800456 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:51.800422 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:51.904167 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:51.901013 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:51.965947 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:51.965920 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" Sep 15 07:42:51.971274 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:51.971249 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:52.078804 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.078770 2552 generic.go:356] "Generic (PLEG): container finished" podID="7a9aa094-c37b-4f88-a7fc-4a27355cfe8c" containerID="63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30" exitCode=0 Sep 15 07:42:52.078804 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.078809 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" event={"ID":"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c","Type":"ContainerDied","Data":"63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30"} Sep 15 07:42:52.079083 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.078824 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" Sep 15 07:42:52.079083 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.078851 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx" event={"ID":"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c","Type":"ContainerDied","Data":"cfb79d0296497213ddbc871c4b793b5bd18e272589ba9fc69f7be5b4958020a8"} Sep 15 07:42:52.079083 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.078896 2552 scope.go:117] "RemoveContainer" containerID="63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30" Sep 15 07:42:52.084211 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.084190 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-959r6\" (UniqueName: \"kubernetes.io/projected/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c-kube-api-access-959r6\") pod \"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c\" (UID: \"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c\") " Sep 15 07:42:52.084328 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.084224 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c-plugins-config-volume\") pod \"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c\" (UID: \"7a9aa094-c37b-4f88-a7fc-4a27355cfe8c\") " Sep 15 07:42:52.084658 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.084638 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a9aa094-c37b-4f88-a7fc-4a27355cfe8c" (UID: "7a9aa094-c37b-4f88-a7fc-4a27355cfe8c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:42:52.086587 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.086564 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c-kube-api-access-959r6" (OuterVolumeSpecName: "kube-api-access-959r6") pod "7a9aa094-c37b-4f88-a7fc-4a27355cfe8c" (UID: "7a9aa094-c37b-4f88-a7fc-4a27355cfe8c"). InnerVolumeSpecName "kube-api-access-959r6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:42:52.088513 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.088495 2552 scope.go:117] "RemoveContainer" containerID="63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30" Sep 15 07:42:52.088779 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:42:52.088759 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30\": container with ID starting with 63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30 not found: ID does not exist" containerID="63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30" Sep 15 07:42:52.088848 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.088784 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30"} err="failed to get container status \"63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30\": rpc error: code = NotFound desc = could not find container \"63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30\": container with ID starting with 63da659d44c150642681f5900213951ef1618d6f7a29ad34acfb818a1eaa3e30 not found: ID does not exist" Sep 15 07:42:52.185776 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.185743 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-959r6\" (UniqueName: \"kubernetes.io/projected/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c-kube-api-access-959r6\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:42:52.185776 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.185771 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:42:52.292076 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.292038 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:52.400713 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.400685 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx"] Sep 15 07:42:52.404401 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:52.404376 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ee0096-f7b6cb579-grrpx"] Sep 15 07:42:53.032338 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:53.032305 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:53.474453 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:53.474429 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a9aa094-c37b-4f88-a7fc-4a27355cfe8c" path="/var/lib/kubelet/pods/7a9aa094-c37b-4f88-a7fc-4a27355cfe8c/volumes" Sep 15 07:42:53.769448 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:53.769364 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:54.676429 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:54.676396 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:55.406284 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:55.406249 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:56.102935 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.102904 2552 generic.go:356] "Generic (PLEG): container finished" podID="04634f3b-130b-4ad0-8bee-78f5ce8a6e53" containerID="d02f78dc4a2b85408fa124a42e146fb9969eaec0d6670842eae6a832ef846e82" exitCode=0 Sep 15 07:42:56.103293 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.102957 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" event={"ID":"04634f3b-130b-4ad0-8bee-78f5ce8a6e53","Type":"ContainerDied","Data":"d02f78dc4a2b85408fa124a42e146fb9969eaec0d6670842eae6a832ef846e82"} Sep 15 07:42:56.159184 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.159128 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:42:56.230529 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.230497 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-workload-socket\") pod \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " Sep 15 07:42:56.230673 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.230593 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-envoy\") pod \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " Sep 15 07:42:56.230673 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.230621 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istiod-ca-cert\") pod \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " Sep 15 07:42:56.230673 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.230641 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-workload-certs\") pod \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " Sep 15 07:42:56.230673 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.230664 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vxt6\" (UniqueName: \"kubernetes.io/projected/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-kube-api-access-7vxt6\") pod \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " Sep 15 07:42:56.230906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.230701 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-credential-socket\") pod \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " Sep 15 07:42:56.230906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.230731 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-token\") pod \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " Sep 15 07:42:56.230906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.230776 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "04634f3b-130b-4ad0-8bee-78f5ce8a6e53" (UID: "04634f3b-130b-4ad0-8bee-78f5ce8a6e53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:42:56.230906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.230810 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-podinfo\") pod \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " Sep 15 07:42:56.230906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.230846 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-data\") pod \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\" (UID: \"04634f3b-130b-4ad0-8bee-78f5ce8a6e53\") " Sep 15 07:42:56.231177 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.230984 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "04634f3b-130b-4ad0-8bee-78f5ce8a6e53" (UID: "04634f3b-130b-4ad0-8bee-78f5ce8a6e53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:42:56.231177 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.231067 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "04634f3b-130b-4ad0-8bee-78f5ce8a6e53" (UID: "04634f3b-130b-4ad0-8bee-78f5ce8a6e53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:42:56.231308 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.231257 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "04634f3b-130b-4ad0-8bee-78f5ce8a6e53" (UID: "04634f3b-130b-4ad0-8bee-78f5ce8a6e53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:42:56.231308 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.231270 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-data" (OuterVolumeSpecName: "istio-data") pod "04634f3b-130b-4ad0-8bee-78f5ce8a6e53" (UID: "04634f3b-130b-4ad0-8bee-78f5ce8a6e53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:42:56.231415 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.231379 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istiod-ca-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:42:56.231415 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.231393 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-workload-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:42:56.231415 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.231402 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-credential-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:42:56.231415 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.231411 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-data\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:42:56.231628 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.231419 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-workload-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:42:56.233001 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.232979 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-token" (OuterVolumeSpecName: "istio-token") pod "04634f3b-130b-4ad0-8bee-78f5ce8a6e53" (UID: "04634f3b-130b-4ad0-8bee-78f5ce8a6e53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:42:56.233123 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.233073 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "04634f3b-130b-4ad0-8bee-78f5ce8a6e53" (UID: "04634f3b-130b-4ad0-8bee-78f5ce8a6e53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 07:42:56.233263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.233244 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-kube-api-access-7vxt6" (OuterVolumeSpecName: "kube-api-access-7vxt6") pod "04634f3b-130b-4ad0-8bee-78f5ce8a6e53" (UID: "04634f3b-130b-4ad0-8bee-78f5ce8a6e53"). InnerVolumeSpecName "kube-api-access-7vxt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:42:56.233319 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.233272 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "04634f3b-130b-4ad0-8bee-78f5ce8a6e53" (UID: "04634f3b-130b-4ad0-8bee-78f5ce8a6e53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:42:56.332091 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.332066 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vxt6\" (UniqueName: \"kubernetes.io/projected/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-kube-api-access-7vxt6\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:42:56.332091 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.332090 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-token\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:42:56.332232 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.332099 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-podinfo\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:42:56.332232 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.332107 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04634f3b-130b-4ad0-8bee-78f5ce8a6e53-istio-envoy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:42:56.473435 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.473341 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:56.574145 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:56.574108 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:57.038332 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:57.038301 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:57.113149 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:57.113123 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" Sep 15 07:42:57.113568 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:57.113158 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk" event={"ID":"04634f3b-130b-4ad0-8bee-78f5ce8a6e53","Type":"ContainerDied","Data":"d109a9a276a77822ed7a73d0838d532f7e92862d78e2de7390b20dde0d2d8d48"} Sep 15 07:42:57.113568 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:57.113223 2552 scope.go:117] "RemoveContainer" containerID="d02f78dc4a2b85408fa124a42e146fb9969eaec0d6670842eae6a832ef846e82" Sep 15 07:42:57.137459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:57.137432 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk"] Sep 15 07:42:57.141550 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:57.141528 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-c3abd1-openshift-default-585df76644-lzcvk"] Sep 15 07:42:57.475500 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:57.475476 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04634f3b-130b-4ad0-8bee-78f5ce8a6e53" path="/var/lib/kubelet/pods/04634f3b-130b-4ad0-8bee-78f5ce8a6e53/volumes" Sep 15 07:42:58.267825 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:58.267786 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:58.367613 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:58.367588 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:58.513232 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:58.513202 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:59.115377 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:59.115347 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:42:59.484926 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:42:59.484894 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:00.057719 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:00.057688 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:00.755408 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:00.755373 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:01.040417 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:01.040334 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:01.514956 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:01.514897 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:02.334062 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:02.334009 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:03.031598 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:03.031558 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:03.884697 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:03.884664 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:04.623178 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:04.623147 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:09.339769 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:09.339736 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:09.359245 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:09.359219 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945"] Sep 15 07:43:09.360756 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:09.360726 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" podUID="868173b2-89ca-4d72-8d5b-c35440bbdd92" containerName="payload-pre-processing" containerID="cri-o://7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69" gracePeriod=30 Sep 15 07:43:09.633232 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:09.633204 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" Sep 15 07:43:09.763368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:09.763339 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7zft\" (UniqueName: \"kubernetes.io/projected/868173b2-89ca-4d72-8d5b-c35440bbdd92-kube-api-access-k7zft\") pod \"868173b2-89ca-4d72-8d5b-c35440bbdd92\" (UID: \"868173b2-89ca-4d72-8d5b-c35440bbdd92\") " Sep 15 07:43:09.763552 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:09.763381 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/868173b2-89ca-4d72-8d5b-c35440bbdd92-plugins-config-volume\") pod \"868173b2-89ca-4d72-8d5b-c35440bbdd92\" (UID: \"868173b2-89ca-4d72-8d5b-c35440bbdd92\") " Sep 15 07:43:09.763823 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:09.763792 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/868173b2-89ca-4d72-8d5b-c35440bbdd92-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "868173b2-89ca-4d72-8d5b-c35440bbdd92" (UID: "868173b2-89ca-4d72-8d5b-c35440bbdd92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:43:09.765479 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:09.765459 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/868173b2-89ca-4d72-8d5b-c35440bbdd92-kube-api-access-k7zft" (OuterVolumeSpecName: "kube-api-access-k7zft") pod "868173b2-89ca-4d72-8d5b-c35440bbdd92" (UID: "868173b2-89ca-4d72-8d5b-c35440bbdd92"). InnerVolumeSpecName "kube-api-access-k7zft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:43:09.864848 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:09.864759 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7zft\" (UniqueName: \"kubernetes.io/projected/868173b2-89ca-4d72-8d5b-c35440bbdd92-kube-api-access-k7zft\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:43:09.864848 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:09.864800 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/868173b2-89ca-4d72-8d5b-c35440bbdd92-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:43:10.187625 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:10.187530 2552 generic.go:356] "Generic (PLEG): container finished" podID="868173b2-89ca-4d72-8d5b-c35440bbdd92" containerID="7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69" exitCode=0 Sep 15 07:43:10.187625 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:10.187574 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" Sep 15 07:43:10.187625 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:10.187597 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" event={"ID":"868173b2-89ca-4d72-8d5b-c35440bbdd92","Type":"ContainerDied","Data":"7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69"} Sep 15 07:43:10.187932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:10.187673 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945" event={"ID":"868173b2-89ca-4d72-8d5b-c35440bbdd92","Type":"ContainerDied","Data":"19653a11ee2aee3fd8ee758c6d8da440be66eaa4802d36446e37c80221df6faa"} Sep 15 07:43:10.187932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:10.187700 2552 scope.go:117] "RemoveContainer" containerID="7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69" Sep 15 07:43:10.196641 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:10.196624 2552 scope.go:117] "RemoveContainer" containerID="7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69" Sep 15 07:43:10.196921 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:43:10.196897 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69\": container with ID starting with 7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69 not found: ID does not exist" containerID="7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69" Sep 15 07:43:10.197009 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:10.196928 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69"} err="failed to get container status \"7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69\": rpc error: code = NotFound desc = could not find container \"7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69\": container with ID starting with 7f966c27dfea7f2f17de306b910d94330a589d3914caf4608abdb2230b97fc69 not found: ID does not exist" Sep 15 07:43:10.207222 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:10.207198 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945"] Sep 15 07:43:10.211275 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:10.211250 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-fcd453-86b7fb78c4-jn945"] Sep 15 07:43:11.281650 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:11.281622 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:11.475088 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:11.475058 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="868173b2-89ca-4d72-8d5b-c35440bbdd92" path="/var/lib/kubelet/pods/868173b2-89ca-4d72-8d5b-c35440bbdd92/volumes" Sep 15 07:43:11.611369 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:11.611275 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:12.308525 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:12.308493 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:13.139801 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:13.139764 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:13.818523 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:13.818462 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:16.115328 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:16.115279 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:16.233339 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:16.233307 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:16.398185 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:16.398104 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r"] Sep 15 07:43:16.398335 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:16.398305 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" containerID="cri-o://c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858" gracePeriod=30 Sep 15 07:43:16.441116 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:16.441044 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:16.922173 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:16.922143 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:17.219887 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.219790 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr"] Sep 15 07:43:17.220404 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.220388 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a9aa094-c37b-4f88-a7fc-4a27355cfe8c" containerName="payload-processing" Sep 15 07:43:17.220458 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.220408 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a9aa094-c37b-4f88-a7fc-4a27355cfe8c" containerName="payload-processing" Sep 15 07:43:17.220458 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.220424 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="868173b2-89ca-4d72-8d5b-c35440bbdd92" containerName="payload-pre-processing" Sep 15 07:43:17.220458 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.220433 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="868173b2-89ca-4d72-8d5b-c35440bbdd92" containerName="payload-pre-processing" Sep 15 07:43:17.220566 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.220476 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04634f3b-130b-4ad0-8bee-78f5ce8a6e53" containerName="istio-proxy" Sep 15 07:43:17.220566 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.220485 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="04634f3b-130b-4ad0-8bee-78f5ce8a6e53" containerName="istio-proxy" Sep 15 07:43:17.220638 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.220578 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="04634f3b-130b-4ad0-8bee-78f5ce8a6e53" containerName="istio-proxy" Sep 15 07:43:17.220638 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.220594 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a9aa094-c37b-4f88-a7fc-4a27355cfe8c" containerName="payload-processing" Sep 15 07:43:17.220638 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.220603 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="868173b2-89ca-4d72-8d5b-c35440bbdd92" containerName="payload-pre-processing" Sep 15 07:43:17.225159 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.225133 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.229066 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.229045 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-04464b-openshift-default-dockercfg-8fk6k\"" Sep 15 07:43:17.235941 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.235918 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr"] Sep 15 07:43:17.236319 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.236285 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-workload-socket\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.236414 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.236319 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwhrl\" (UniqueName: \"kubernetes.io/projected/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-kube-api-access-vwhrl\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.236414 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.236383 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-token\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.236518 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.236413 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-envoy\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.236518 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.236469 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-credential-socket\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.236518 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.236510 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-workload-certs\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.236720 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.236569 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-podinfo\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.236720 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.236613 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-data\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.236855 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.236728 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istiod-ca-cert\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.246233 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.246208 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:17.338038 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.337979 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istiod-ca-cert\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.338238 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.338069 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-workload-socket\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.338238 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.338104 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwhrl\" (UniqueName: \"kubernetes.io/projected/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-kube-api-access-vwhrl\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.338238 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.338156 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-token\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.338238 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.338188 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-envoy\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.338463 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.338244 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-credential-socket\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.338463 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.338268 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-workload-certs\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.338463 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.338337 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-podinfo\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.338463 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.338382 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-data\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.338902 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.338834 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-data\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.339501 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.339472 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istiod-ca-cert\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.339742 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.339711 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-workload-socket\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.340779 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.340737 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-workload-certs\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.340974 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.340947 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-credential-socket\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.342639 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.342620 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-envoy\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.343581 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.343560 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-podinfo\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.347089 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.347067 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-token\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.347323 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.347301 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwhrl\" (UniqueName: \"kubernetes.io/projected/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-kube-api-access-vwhrl\") pod \"e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.538434 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.538403 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:17.702355 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.702319 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:17.816255 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.816226 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr"] Sep 15 07:43:17.818174 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:43:17.818149 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5e2d6c_ff5c_49e5_b6ff_10fcae0c5cc0.slice/crio-c5cc37b70082536e99c3f076f8ffc3fe8c833dec9ac0ae4d551ee9822f0b8823 WatchSource:0}: Error finding container c5cc37b70082536e99c3f076f8ffc3fe8c833dec9ac0ae4d551ee9822f0b8823: Status 404 returned error can't find the container with id c5cc37b70082536e99c3f076f8ffc3fe8c833dec9ac0ae4d551ee9822f0b8823 Sep 15 07:43:17.820971 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.820941 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:43:17.821039 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.821005 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:43:17.821077 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.821039 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:43:17.833257 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:17.833234 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:18.239587 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:18.239553 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" event={"ID":"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0","Type":"ContainerStarted","Data":"0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5"} Sep 15 07:43:18.240078 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:18.239617 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" event={"ID":"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0","Type":"ContainerStarted","Data":"c5cc37b70082536e99c3f076f8ffc3fe8c833dec9ac0ae4d551ee9822f0b8823"} Sep 15 07:43:18.261602 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:18.261556 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" podStartSLOduration=1.261542432 podStartE2EDuration="1.261542432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:43:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:43:18.26036706 +0000 UTC m=+1479.241529230" watchObservedRunningTime="2026-09-15 07:43:18.261542432 +0000 UTC m=+1479.242704598" Sep 15 07:43:18.538956 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:18.538858 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:18.544992 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:18.544966 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:19.245262 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:19.245232 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:19.246296 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:19.246275 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:19.627036 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:19.626957 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:20.259796 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:20.259764 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:20.343224 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:20.343189 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:20.443508 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:20.443477 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:21.303642 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.303609 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:21.660668 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.660646 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:43:21.681552 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.681523 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff0ba423-6d42-423e-a11b-327dca1e6935-istiod-ca-cert\") pod \"ff0ba423-6d42-423e-a11b-327dca1e6935\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " Sep 15 07:43:21.681738 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.681570 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-workload-certs\") pod \"ff0ba423-6d42-423e-a11b-327dca1e6935\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " Sep 15 07:43:21.681738 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.681605 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-token\") pod \"ff0ba423-6d42-423e-a11b-327dca1e6935\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " Sep 15 07:43:21.681738 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.681645 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-envoy\") pod \"ff0ba423-6d42-423e-a11b-327dca1e6935\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " Sep 15 07:43:21.681738 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.681729 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msfk4\" (UniqueName: \"kubernetes.io/projected/ff0ba423-6d42-423e-a11b-327dca1e6935-kube-api-access-msfk4\") pod \"ff0ba423-6d42-423e-a11b-327dca1e6935\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " Sep 15 07:43:21.681998 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.681825 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-workload-socket\") pod \"ff0ba423-6d42-423e-a11b-327dca1e6935\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " Sep 15 07:43:21.681998 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.681854 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-data\") pod \"ff0ba423-6d42-423e-a11b-327dca1e6935\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " Sep 15 07:43:21.681998 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.681908 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-credential-socket\") pod \"ff0ba423-6d42-423e-a11b-327dca1e6935\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " Sep 15 07:43:21.681998 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.681944 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-podinfo\") pod \"ff0ba423-6d42-423e-a11b-327dca1e6935\" (UID: \"ff0ba423-6d42-423e-a11b-327dca1e6935\") " Sep 15 07:43:21.683395 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.682747 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ff0ba423-6d42-423e-a11b-327dca1e6935" (UID: "ff0ba423-6d42-423e-a11b-327dca1e6935"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:43:21.683395 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.683327 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-data" (OuterVolumeSpecName: "istio-data") pod "ff0ba423-6d42-423e-a11b-327dca1e6935" (UID: "ff0ba423-6d42-423e-a11b-327dca1e6935"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:43:21.683583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.683524 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ff0ba423-6d42-423e-a11b-327dca1e6935" (UID: "ff0ba423-6d42-423e-a11b-327dca1e6935"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:43:21.683715 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.683676 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff0ba423-6d42-423e-a11b-327dca1e6935-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ff0ba423-6d42-423e-a11b-327dca1e6935" (UID: "ff0ba423-6d42-423e-a11b-327dca1e6935"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:43:21.683919 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.683815 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ff0ba423-6d42-423e-a11b-327dca1e6935" (UID: "ff0ba423-6d42-423e-a11b-327dca1e6935"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:43:21.684616 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.684586 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ff0ba423-6d42-423e-a11b-327dca1e6935" (UID: "ff0ba423-6d42-423e-a11b-327dca1e6935"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 07:43:21.684720 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.684640 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff0ba423-6d42-423e-a11b-327dca1e6935" (UID: "ff0ba423-6d42-423e-a11b-327dca1e6935"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:43:21.685038 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.685006 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ff0ba423-6d42-423e-a11b-327dca1e6935" (UID: "ff0ba423-6d42-423e-a11b-327dca1e6935"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:43:21.686060 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.686031 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff0ba423-6d42-423e-a11b-327dca1e6935-kube-api-access-msfk4" (OuterVolumeSpecName: "kube-api-access-msfk4") pod "ff0ba423-6d42-423e-a11b-327dca1e6935" (UID: "ff0ba423-6d42-423e-a11b-327dca1e6935"). InnerVolumeSpecName "kube-api-access-msfk4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:43:21.783320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.783294 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-workload-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:43:21.783320 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.783319 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-data\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:43:21.783489 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.783329 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-credential-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:43:21.783489 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.783338 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-podinfo\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:43:21.783489 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.783346 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff0ba423-6d42-423e-a11b-327dca1e6935-istiod-ca-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:43:21.783489 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.783354 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-workload-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:43:21.783489 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.783362 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-token\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:43:21.783489 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.783369 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff0ba423-6d42-423e-a11b-327dca1e6935-istio-envoy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:43:21.783489 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:21.783378 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msfk4\" (UniqueName: \"kubernetes.io/projected/ff0ba423-6d42-423e-a11b-327dca1e6935-kube-api-access-msfk4\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:43:22.149311 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.149276 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:22.263618 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.263584 2552 generic.go:356] "Generic (PLEG): container finished" podID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerID="c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858" exitCode=0 Sep 15 07:43:22.263800 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.263651 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" Sep 15 07:43:22.263800 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.263664 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" event={"ID":"ff0ba423-6d42-423e-a11b-327dca1e6935","Type":"ContainerDied","Data":"c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858"} Sep 15 07:43:22.263800 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.263719 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r" event={"ID":"ff0ba423-6d42-423e-a11b-327dca1e6935","Type":"ContainerDied","Data":"2123cec6187eca9ba00e641818b6ec92468c908c88ec3bfeb3016b6f3fe5948e"} Sep 15 07:43:22.263800 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.263745 2552 scope.go:117] "RemoveContainer" containerID="c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858" Sep 15 07:43:22.273595 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.273578 2552 scope.go:117] "RemoveContainer" containerID="23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7" Sep 15 07:43:22.281429 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.281408 2552 scope.go:117] "RemoveContainer" containerID="c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858" Sep 15 07:43:22.281666 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:43:22.281650 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858\": container with ID starting with c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858 not found: ID does not exist" containerID="c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858" Sep 15 07:43:22.281711 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.281673 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858"} err="failed to get container status \"c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858\": rpc error: code = NotFound desc = could not find container \"c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858\": container with ID starting with c3e4cef5f82a0d5edb5078128bec807e6f34902ccb5e44061db2fb348fbcd858 not found: ID does not exist" Sep 15 07:43:22.281711 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.281687 2552 scope.go:117] "RemoveContainer" containerID="23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7" Sep 15 07:43:22.281954 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:43:22.281938 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7\": container with ID starting with 23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7 not found: ID does not exist" containerID="23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7" Sep 15 07:43:22.282016 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.281957 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7"} err="failed to get container status \"23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7\": rpc error: code = NotFound desc = could not find container \"23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7\": container with ID starting with 23d375af564cc79697974d968e49b78ffdd6ebcfef1240a3818a0e8788fce5d7 not found: ID does not exist" Sep 15 07:43:22.286510 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.286490 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r"] Sep 15 07:43:22.292195 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:22.292175 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-fcd453-openshift-default-6758677554-78r2r"] Sep 15 07:43:23.039969 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:23.039939 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:23.475755 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:23.475725 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" path="/var/lib/kubelet/pods/ff0ba423-6d42-423e-a11b-327dca1e6935/volumes" Sep 15 07:43:23.818380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:23.818298 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:24.665161 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:24.665122 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:24.765624 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:24.765593 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:24.867540 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:24.867510 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:25.128844 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.128815 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:25.207542 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.207511 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:25.265737 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.265707 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5"] Sep 15 07:43:25.266151 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.266137 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" Sep 15 07:43:25.266204 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.266154 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" Sep 15 07:43:25.266204 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.266176 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" Sep 15 07:43:25.266204 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.266186 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" Sep 15 07:43:25.266350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.266262 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" Sep 15 07:43:25.270691 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.270671 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" Sep 15 07:43:25.273194 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.273175 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-04464b-dockercfg-ndsf9\"" Sep 15 07:43:25.273290 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.273184 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-04464b\"" Sep 15 07:43:25.282434 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.282413 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5"] Sep 15 07:43:25.317392 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.317366 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7qxw\" (UniqueName: \"kubernetes.io/projected/6685c800-6252-4098-82c4-ae1f7419e312-kube-api-access-g7qxw\") pod \"payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5\" (UID: \"6685c800-6252-4098-82c4-ae1f7419e312\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" Sep 15 07:43:25.317496 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.317406 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6685c800-6252-4098-82c4-ae1f7419e312-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5\" (UID: \"6685c800-6252-4098-82c4-ae1f7419e312\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" Sep 15 07:43:25.418763 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.418679 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7qxw\" (UniqueName: \"kubernetes.io/projected/6685c800-6252-4098-82c4-ae1f7419e312-kube-api-access-g7qxw\") pod \"payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5\" (UID: \"6685c800-6252-4098-82c4-ae1f7419e312\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" Sep 15 07:43:25.418763 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.418724 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6685c800-6252-4098-82c4-ae1f7419e312-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5\" (UID: \"6685c800-6252-4098-82c4-ae1f7419e312\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" Sep 15 07:43:25.419433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.419415 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6685c800-6252-4098-82c4-ae1f7419e312-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5\" (UID: \"6685c800-6252-4098-82c4-ae1f7419e312\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" Sep 15 07:43:25.426438 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.426419 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7qxw\" (UniqueName: \"kubernetes.io/projected/6685c800-6252-4098-82c4-ae1f7419e312-kube-api-access-g7qxw\") pod \"payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5\" (UID: \"6685c800-6252-4098-82c4-ae1f7419e312\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" Sep 15 07:43:25.582483 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.582450 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" Sep 15 07:43:25.855232 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:25.855208 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5"] Sep 15 07:43:25.857425 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:43:25.857394 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6685c800_6252_4098_82c4_ae1f7419e312.slice/crio-705852c62a52697fedd723e247b12b1fc310dfffc3e30846ebd1ac6952e6b8a5 WatchSource:0}: Error finding container 705852c62a52697fedd723e247b12b1fc310dfffc3e30846ebd1ac6952e6b8a5: Status 404 returned error can't find the container with id 705852c62a52697fedd723e247b12b1fc310dfffc3e30846ebd1ac6952e6b8a5 Sep 15 07:43:26.295695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:26.295656 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" event={"ID":"6685c800-6252-4098-82c4-ae1f7419e312","Type":"ContainerStarted","Data":"3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98"} Sep 15 07:43:26.295935 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:26.295723 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" event={"ID":"6685c800-6252-4098-82c4-ae1f7419e312","Type":"ContainerStarted","Data":"705852c62a52697fedd723e247b12b1fc310dfffc3e30846ebd1ac6952e6b8a5"} Sep 15 07:43:26.295935 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:26.295756 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" Sep 15 07:43:26.313785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:26.313738 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" podStartSLOduration=1.313724123 podStartE2EDuration="1.313724123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:43:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:43:26.311254883 +0000 UTC m=+1487.292417049" watchObservedRunningTime="2026-09-15 07:43:26.313724123 +0000 UTC m=+1487.294886289" Sep 15 07:43:33.021306 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:33.021274 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:37.305628 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:37.305544 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" Sep 15 07:43:39.585058 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:39.585026 2552 scope.go:117] "RemoveContainer" containerID="1bef0402bc6d84535c2269bec8bb54349d6161f429326e0b96306cd0541c5411" Sep 15 07:43:41.079673 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:41.079639 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:43:41.083049 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:41.083026 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:43:41.086702 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:41.086683 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:43:41.089695 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:41.089678 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:43:41.798809 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:41.798778 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:41.900195 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:41.900165 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:46.746032 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:46.745994 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:47.573527 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:47.573485 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:48.248987 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:48.248948 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:49.898490 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:49.898457 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:49.999653 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:49.999619 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:50.331448 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:50.331418 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:50.689614 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:50.689525 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:50.789481 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:50.789449 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:50.978832 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:50.978804 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6"] Sep 15 07:43:50.979323 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:50.979308 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff0ba423-6d42-423e-a11b-327dca1e6935" containerName="istio-proxy" Sep 15 07:43:50.983979 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:50.983960 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:50.986913 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:50.986887 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-e3a482-openshift-default-dockercfg-qj5js\"" Sep 15 07:43:50.997453 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:50.997432 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6"] Sep 15 07:43:50.999500 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:50.999475 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:51.069512 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.069405 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-workload-socket\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.069512 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.069459 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-token\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.069512 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.069494 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-workload-certs\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.069812 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.069526 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-data\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.069812 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.069558 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-credential-socket\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.069812 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.069594 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istiod-ca-cert\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.069812 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.069652 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq7jw\" (UniqueName: \"kubernetes.io/projected/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-kube-api-access-qq7jw\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.069812 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.069732 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-podinfo\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.069812 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.069772 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-envoy\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.171097 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.171056 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istiod-ca-cert\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.171302 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.171137 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq7jw\" (UniqueName: \"kubernetes.io/projected/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-kube-api-access-qq7jw\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.171302 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.171238 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-podinfo\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.171302 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.171282 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-envoy\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.171475 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.171336 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-workload-socket\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.171475 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.171362 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-token\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.171475 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.171399 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-workload-certs\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.171475 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.171433 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-data\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.171475 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.171465 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-credential-socket\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.172139 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.171792 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-workload-certs\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.172139 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.172012 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istiod-ca-cert\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.172139 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.172042 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-workload-socket\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.172139 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.172094 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-credential-socket\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.172139 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.172111 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-data\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.174307 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.174277 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-envoy\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.174572 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.174554 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-podinfo\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.179174 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.179153 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-token\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.179704 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.179678 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq7jw\" (UniqueName: \"kubernetes.io/projected/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-kube-api-access-qq7jw\") pod \"e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.282505 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.282421 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:51.296923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.296899 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:51.592609 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.592501 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:51.594105 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.594071 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6"] Sep 15 07:43:51.596445 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:43:51.596415 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e01a9e_b2d5_4ee9_97c5_157c8d181d32.slice/crio-4c452f7fcefc953426a621c2f7c79bf61e3a7658e37c2ea7b5490279ccb21938 WatchSource:0}: Error finding container 4c452f7fcefc953426a621c2f7c79bf61e3a7658e37c2ea7b5490279ccb21938: Status 404 returned error can't find the container with id 4c452f7fcefc953426a621c2f7c79bf61e3a7658e37c2ea7b5490279ccb21938 Sep 15 07:43:51.598757 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.598717 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:43:51.598886 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.598801 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:43:51.598886 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:51.598844 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:43:52.437849 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:52.437806 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" event={"ID":"03e01a9e-b2d5-4ee9-97c5-157c8d181d32","Type":"ContainerStarted","Data":"957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf"} Sep 15 07:43:52.438234 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:52.437892 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" event={"ID":"03e01a9e-b2d5-4ee9-97c5-157c8d181d32","Type":"ContainerStarted","Data":"4c452f7fcefc953426a621c2f7c79bf61e3a7658e37c2ea7b5490279ccb21938"} Sep 15 07:43:52.478525 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:52.478478 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" podStartSLOduration=2.478462407 podStartE2EDuration="2.478462407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:43:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:43:52.477188441 +0000 UTC m=+1513.458350612" watchObservedRunningTime="2026-09-15 07:43:52.478462407 +0000 UTC m=+1513.459624573" Sep 15 07:43:53.301339 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:53.301299 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:53.307014 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:53.306985 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:53.444178 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:53.444145 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:53.445169 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:53.445152 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:43:53.748927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:53.748888 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:54.448140 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:54.448086 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:54.451939 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:54.451912 2552 generic.go:356] "Generic (PLEG): container finished" podID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerID="0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5" exitCode=0 Sep 15 07:43:54.452074 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:54.451987 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" event={"ID":"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0","Type":"ContainerDied","Data":"0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5"} Sep 15 07:43:54.452467 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:54.452446 2552 scope.go:117] "RemoveContainer" containerID="0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5" Sep 15 07:43:54.453684 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:54.453658 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:43:54.453741 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:54.453729 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:43:54.453779 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:54.453761 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:43:54.542484 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:54.542436 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:55.458878 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:55.458834 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" event={"ID":"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0","Type":"ContainerStarted","Data":"014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a"} Sep 15 07:43:55.538716 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:55.538678 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:43:55.538954 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:55.538930 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 15 07:43:55.539014 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:55.538995 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 15 07:43:56.539458 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:56.539425 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 15 07:43:56.539852 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:56.539486 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 15 07:43:56.807265 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:56.807171 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:57.539466 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:57.539433 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 15 07:43:57.539674 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:57.539510 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 15 07:43:58.155753 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:58.155717 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:58.256890 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:58.256819 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:58.363151 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:58.363095 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:58.538819 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:58.538781 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 15 07:43:58.539014 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:58.538840 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 15 07:43:59.088464 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.088347 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:59.169358 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.169307 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:59.539716 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.539684 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 15 07:43:59.539898 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.539735 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 15 07:43:59.688767 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.688717 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw"] Sep 15 07:43:59.702490 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.702465 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw"] Sep 15 07:43:59.702656 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.702567 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" Sep 15 07:43:59.704936 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.704914 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-e3a482-dockercfg-jmlpd\"" Sep 15 07:43:59.705065 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.704920 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-e3a482\"" Sep 15 07:43:59.760276 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.760242 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmcjk\" (UniqueName: \"kubernetes.io/projected/f0566e78-fce4-4725-b1e3-dafbe7ea164f-kube-api-access-xmcjk\") pod \"payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw\" (UID: \"f0566e78-fce4-4725-b1e3-dafbe7ea164f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" Sep 15 07:43:59.760467 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.760433 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0566e78-fce4-4725-b1e3-dafbe7ea164f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw\" (UID: \"f0566e78-fce4-4725-b1e3-dafbe7ea164f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" Sep 15 07:43:59.822561 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.822466 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:43:59.861231 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.861200 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0566e78-fce4-4725-b1e3-dafbe7ea164f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw\" (UID: \"f0566e78-fce4-4725-b1e3-dafbe7ea164f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" Sep 15 07:43:59.861371 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.861264 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmcjk\" (UniqueName: \"kubernetes.io/projected/f0566e78-fce4-4725-b1e3-dafbe7ea164f-kube-api-access-xmcjk\") pod \"payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw\" (UID: \"f0566e78-fce4-4725-b1e3-dafbe7ea164f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" Sep 15 07:43:59.861893 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.861849 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0566e78-fce4-4725-b1e3-dafbe7ea164f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw\" (UID: \"f0566e78-fce4-4725-b1e3-dafbe7ea164f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" Sep 15 07:43:59.873576 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.873543 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmcjk\" (UniqueName: \"kubernetes.io/projected/f0566e78-fce4-4725-b1e3-dafbe7ea164f-kube-api-access-xmcjk\") pod \"payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw\" (UID: \"f0566e78-fce4-4725-b1e3-dafbe7ea164f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" Sep 15 07:43:59.923035 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:43:59.922999 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:44:00.013954 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:00.013916 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" Sep 15 07:44:00.287929 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:00.287847 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw"] Sep 15 07:44:00.294026 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:44:00.293993 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0566e78_fce4_4725_b1e3_dafbe7ea164f.slice/crio-2ce79f596325cd35b0a39d92d822be1133cb3079e3f1cf6c20c193acf19e1f0c WatchSource:0}: Error finding container 2ce79f596325cd35b0a39d92d822be1133cb3079e3f1cf6c20c193acf19e1f0c: Status 404 returned error can't find the container with id 2ce79f596325cd35b0a39d92d822be1133cb3079e3f1cf6c20c193acf19e1f0c Sep 15 07:44:00.519613 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:00.519574 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" event={"ID":"f0566e78-fce4-4725-b1e3-dafbe7ea164f","Type":"ContainerStarted","Data":"c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2"} Sep 15 07:44:00.519796 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:00.519732 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" Sep 15 07:44:00.519796 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:00.519775 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" event={"ID":"f0566e78-fce4-4725-b1e3-dafbe7ea164f","Type":"ContainerStarted","Data":"2ce79f596325cd35b0a39d92d822be1133cb3079e3f1cf6c20c193acf19e1f0c"} Sep 15 07:44:00.537395 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:00.537337 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" podStartSLOduration=1.5373199149999999 podStartE2EDuration="1.537319915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:43:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:44:00.534780471 +0000 UTC m=+1521.515942637" watchObservedRunningTime="2026-09-15 07:44:00.537319915 +0000 UTC m=+1521.518482085" Sep 15 07:44:00.539682 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:00.539612 2552 patch_prober.go:28] interesting pod/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Sep 15 07:44:00.539682 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:00.539657 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Sep 15 07:44:01.543660 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:01.543631 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:44:01.544098 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:01.544085 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:44:01.544724 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:01.544706 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:44:11.528009 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:11.527978 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" Sep 15 07:44:21.114252 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:21.114224 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:44:21.643558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:21.643525 2552 generic.go:356] "Generic (PLEG): container finished" podID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerID="957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf" exitCode=0 Sep 15 07:44:21.643755 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:21.643605 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" event={"ID":"03e01a9e-b2d5-4ee9-97c5-157c8d181d32","Type":"ContainerDied","Data":"957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf"} Sep 15 07:44:21.644071 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:21.644054 2552 scope.go:117] "RemoveContainer" containerID="957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf" Sep 15 07:44:21.645207 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:21.645179 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:44:21.645301 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:21.645263 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:44:21.645369 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:21.645311 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:44:21.952429 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:21.952307 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:44:22.651117 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:22.651085 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" event={"ID":"03e01a9e-b2d5-4ee9-97c5-157c8d181d32","Type":"ContainerStarted","Data":"3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758"} Sep 15 07:44:22.660702 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:22.660676 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:44:23.297191 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:23.297155 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:44:23.298421 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:23.298391 2552 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Sep 15 07:44:23.298551 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:23.298452 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Sep 15 07:44:23.324111 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:23.324079 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:44:23.444618 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:23.444582 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:44:24.143815 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:24.143777 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:44:24.297477 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:24.297439 2552 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Sep 15 07:44:24.297675 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:24.297506 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Sep 15 07:44:24.468128 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:24.468039 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:44:24.843676 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:24.843644 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:44:25.297999 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:25.297954 2552 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Sep 15 07:44:25.298465 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:25.298036 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Sep 15 07:44:25.727806 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:25.727779 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:44:26.297681 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:26.297646 2552 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Sep 15 07:44:26.297910 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:26.297705 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Sep 15 07:44:26.409543 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:26.409507 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:44:27.298250 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:27.298214 2552 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Sep 15 07:44:27.298433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:27.298270 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Sep 15 07:44:28.302168 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:28.302136 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:44:28.303107 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:44:28.303089 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:45:00.142346 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.142310 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-27b9d8-29824305-xqhcg"] Sep 15 07:45:00.145931 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.145914 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8"] Sep 15 07:45:00.146046 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.146024 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-27b9d8-29824305-xqhcg" Sep 15 07:45:00.148487 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.148461 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8jpwk\"" Sep 15 07:45:00.149290 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.149275 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8" Sep 15 07:45:00.160013 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.159990 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-27b9d8-29824305-xqhcg" Sep 15 07:45:00.160234 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.160213 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-27b9d8-29824305-xqhcg"] Sep 15 07:45:00.163330 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.163306 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8"] Sep 15 07:45:00.170635 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.170616 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8" Sep 15 07:45:00.171336 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.171311 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr"] Sep 15 07:45:00.175309 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.175276 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5"] Sep 15 07:45:00.175423 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.175406 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr" Sep 15 07:45:00.179783 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.179746 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5" Sep 15 07:45:00.194259 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.194226 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr"] Sep 15 07:45:00.197838 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.197786 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5"] Sep 15 07:45:00.225785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.225719 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr" Sep 15 07:45:00.237701 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.237515 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5" Sep 15 07:45:00.451094 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.450987 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-27b9d8-29824305-xqhcg"] Sep 15 07:45:00.455949 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:45:00.455908 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2be7f1_b12f_4f1d_9b97_086f223850ee.slice/crio-a13ada82d7a4398ae63fefae4066c8c0cbde267f7454d5c83040bc9b1a01e76a WatchSource:0}: Error finding container a13ada82d7a4398ae63fefae4066c8c0cbde267f7454d5c83040bc9b1a01e76a: Status 404 returned error can't find the container with id a13ada82d7a4398ae63fefae4066c8c0cbde267f7454d5c83040bc9b1a01e76a Sep 15 07:45:00.470718 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.470688 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8"] Sep 15 07:45:00.471847 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:45:00.471823 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5898b14d_d6b2_4fd5_b0fb_1557192cab80.slice/crio-af335bc6c10c17c9a82e3b83ce5dbbe38b1ae4ec26ae340bc35cb0f546ec7574 WatchSource:0}: Error finding container af335bc6c10c17c9a82e3b83ce5dbbe38b1ae4ec26ae340bc35cb0f546ec7574: Status 404 returned error can't find the container with id af335bc6c10c17c9a82e3b83ce5dbbe38b1ae4ec26ae340bc35cb0f546ec7574 Sep 15 07:45:00.522990 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.522965 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr"] Sep 15 07:45:00.524935 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:45:00.524909 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5b8698_3bac_43f6_8428_be56b15d8405.slice/crio-42ae4156e79783393129edaa085de011dd5ae749af2ab49b7a2239ddc65a3fe7 WatchSource:0}: Error finding container 42ae4156e79783393129edaa085de011dd5ae749af2ab49b7a2239ddc65a3fe7: Status 404 returned error can't find the container with id 42ae4156e79783393129edaa085de011dd5ae749af2ab49b7a2239ddc65a3fe7 Sep 15 07:45:00.534417 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.534395 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5"] Sep 15 07:45:00.538021 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:45:00.537999 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45a4a587_309f_4e9c_91df_5137e35b3f91.slice/crio-85c36613ba0d1433ed18f92ebf683fca2945af42a5e9d22df05fc78a4487a8b9 WatchSource:0}: Error finding container 85c36613ba0d1433ed18f92ebf683fca2945af42a5e9d22df05fc78a4487a8b9: Status 404 returned error can't find the container with id 85c36613ba0d1433ed18f92ebf683fca2945af42a5e9d22df05fc78a4487a8b9 Sep 15 07:45:00.864972 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.864932 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5" event={"ID":"45a4a587-309f-4e9c-91df-5137e35b3f91","Type":"ContainerStarted","Data":"a7e3797bc46b5b0022ea4856a3fd7bae286f37e743a97e757f6a39ffba6ce297"} Sep 15 07:45:00.865160 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.864995 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5" event={"ID":"45a4a587-309f-4e9c-91df-5137e35b3f91","Type":"ContainerStarted","Data":"85c36613ba0d1433ed18f92ebf683fca2945af42a5e9d22df05fc78a4487a8b9"} Sep 15 07:45:00.866647 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.866623 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr" event={"ID":"1e5b8698-3bac-43f6-8428-be56b15d8405","Type":"ContainerStarted","Data":"8b75957b9f5917f8d712c08646bfc396dde8c19e70418b3e6dcc4df78c08435d"} Sep 15 07:45:00.866745 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.866671 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr" event={"ID":"1e5b8698-3bac-43f6-8428-be56b15d8405","Type":"ContainerStarted","Data":"42ae4156e79783393129edaa085de011dd5ae749af2ab49b7a2239ddc65a3fe7"} Sep 15 07:45:00.868632 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.868600 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8" event={"ID":"5898b14d-d6b2-4fd5-b0fb-1557192cab80","Type":"ContainerStarted","Data":"7f41951b4556a00d4922620494bb22f1d54213d885b04af8b3166a46b1f17ab9"} Sep 15 07:45:00.868734 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.868648 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8" event={"ID":"5898b14d-d6b2-4fd5-b0fb-1557192cab80","Type":"ContainerStarted","Data":"af335bc6c10c17c9a82e3b83ce5dbbe38b1ae4ec26ae340bc35cb0f546ec7574"} Sep 15 07:45:00.870301 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.870267 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-27b9d8-29824305-xqhcg" event={"ID":"3e2be7f1-b12f-4f1d-9b97-086f223850ee","Type":"ContainerStarted","Data":"a13ada82d7a4398ae63fefae4066c8c0cbde267f7454d5c83040bc9b1a01e76a"} Sep 15 07:45:00.881307 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.881259 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5" podStartSLOduration=0.88124764 podStartE2EDuration="881.24764ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:45:00.879529694 +0000 UTC m=+1581.860691866" watchObservedRunningTime="2026-09-15 07:45:00.88124764 +0000 UTC m=+1581.862409866" Sep 15 07:45:00.894575 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.894536 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8" podStartSLOduration=0.894522543 podStartE2EDuration="894.522543ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:45:00.893428729 +0000 UTC m=+1581.874590895" watchObservedRunningTime="2026-09-15 07:45:00.894522543 +0000 UTC m=+1581.875684710" Sep 15 07:45:00.908939 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:00.908894 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr" podStartSLOduration=0.908880279 podStartE2EDuration="908.880279ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:45:00.907609226 +0000 UTC m=+1581.888771390" watchObservedRunningTime="2026-09-15 07:45:00.908880279 +0000 UTC m=+1581.890042438" Sep 15 07:45:01.884198 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:01.884162 2552 generic.go:356] "Generic (PLEG): container finished" podID="3e2be7f1-b12f-4f1d-9b97-086f223850ee" containerID="dba5c75fdec09d88f9a01313f2b7548a5ccee3e17b675ff6802bc50665f92907" exitCode=0 Sep 15 07:45:01.884690 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:01.884250 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-27b9d8-29824305-xqhcg" event={"ID":"3e2be7f1-b12f-4f1d-9b97-086f223850ee","Type":"ContainerDied","Data":"dba5c75fdec09d88f9a01313f2b7548a5ccee3e17b675ff6802bc50665f92907"} Sep 15 07:45:01.885910 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:01.885862 2552 generic.go:356] "Generic (PLEG): container finished" podID="45a4a587-309f-4e9c-91df-5137e35b3f91" containerID="a7e3797bc46b5b0022ea4856a3fd7bae286f37e743a97e757f6a39ffba6ce297" exitCode=0 Sep 15 07:45:01.886012 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:01.885954 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5" event={"ID":"45a4a587-309f-4e9c-91df-5137e35b3f91","Type":"ContainerDied","Data":"a7e3797bc46b5b0022ea4856a3fd7bae286f37e743a97e757f6a39ffba6ce297"} Sep 15 07:45:01.887658 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:01.887634 2552 generic.go:356] "Generic (PLEG): container finished" podID="1e5b8698-3bac-43f6-8428-be56b15d8405" containerID="8b75957b9f5917f8d712c08646bfc396dde8c19e70418b3e6dcc4df78c08435d" exitCode=0 Sep 15 07:45:01.887767 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:01.887733 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr" event={"ID":"1e5b8698-3bac-43f6-8428-be56b15d8405","Type":"ContainerDied","Data":"8b75957b9f5917f8d712c08646bfc396dde8c19e70418b3e6dcc4df78c08435d"} Sep 15 07:45:01.889391 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:01.889370 2552 generic.go:356] "Generic (PLEG): container finished" podID="5898b14d-d6b2-4fd5-b0fb-1557192cab80" containerID="7f41951b4556a00d4922620494bb22f1d54213d885b04af8b3166a46b1f17ab9" exitCode=0 Sep 15 07:45:01.889478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:01.889399 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8" event={"ID":"5898b14d-d6b2-4fd5-b0fb-1557192cab80","Type":"ContainerDied","Data":"7f41951b4556a00d4922620494bb22f1d54213d885b04af8b3166a46b1f17ab9"} Sep 15 07:45:03.047597 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.047569 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5" Sep 15 07:45:03.096470 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.096446 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr" Sep 15 07:45:03.127834 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.127771 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-27b9d8-29824305-xqhcg" Sep 15 07:45:03.130973 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.130950 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8" Sep 15 07:45:03.906087 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.905997 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-27b9d8-29824305-xqhcg" Sep 15 07:45:03.906087 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.906018 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-27b9d8-29824305-xqhcg" event={"ID":"3e2be7f1-b12f-4f1d-9b97-086f223850ee","Type":"ContainerDied","Data":"a13ada82d7a4398ae63fefae4066c8c0cbde267f7454d5c83040bc9b1a01e76a"} Sep 15 07:45:03.906087 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.906070 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a13ada82d7a4398ae63fefae4066c8c0cbde267f7454d5c83040bc9b1a01e76a" Sep 15 07:45:03.907408 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.907387 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5" Sep 15 07:45:03.907530 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.907408 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5" event={"ID":"45a4a587-309f-4e9c-91df-5137e35b3f91","Type":"ContainerDied","Data":"85c36613ba0d1433ed18f92ebf683fca2945af42a5e9d22df05fc78a4487a8b9"} Sep 15 07:45:03.907530 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.907453 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85c36613ba0d1433ed18f92ebf683fca2945af42a5e9d22df05fc78a4487a8b9" Sep 15 07:45:03.908980 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.908950 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr" Sep 15 07:45:03.909090 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.908996 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr" event={"ID":"1e5b8698-3bac-43f6-8428-be56b15d8405","Type":"ContainerDied","Data":"42ae4156e79783393129edaa085de011dd5ae749af2ab49b7a2239ddc65a3fe7"} Sep 15 07:45:03.909090 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.909034 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="42ae4156e79783393129edaa085de011dd5ae749af2ab49b7a2239ddc65a3fe7" Sep 15 07:45:03.910524 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.910505 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8" event={"ID":"5898b14d-d6b2-4fd5-b0fb-1557192cab80","Type":"ContainerDied","Data":"af335bc6c10c17c9a82e3b83ce5dbbe38b1ae4ec26ae340bc35cb0f546ec7574"} Sep 15 07:45:03.910600 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.910540 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af335bc6c10c17c9a82e3b83ce5dbbe38b1ae4ec26ae340bc35cb0f546ec7574" Sep 15 07:45:03.910600 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:03.910512 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8" Sep 15 07:45:07.092979 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:07.092949 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:45:45.752248 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:45.752209 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:45:46.537774 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:46.537737 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:45:59.616755 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:45:59.616723 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:00.446084 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:00.446049 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:01.235717 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:01.235683 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:01.950578 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:01.950545 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:03.065341 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:03.065297 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:03.145381 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:03.145347 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:03.734763 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:03.734720 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:04.518062 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:04.518029 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:05.259895 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:05.259847 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:05.964821 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:05.964791 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:09.245049 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:09.244998 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:14.822013 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:14.821971 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8"] Sep 15 07:46:14.823622 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:14.823594 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:14.824335 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:14.824315 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b21d1ff2-29824305-zfxw8"] Sep 15 07:46:14.845611 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:14.845584 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn"] Sep 15 07:46:14.845884 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:14.845821 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" podUID="03157279-7fb5-48b9-9fbf-06ee72000524" containerName="payload-processing" containerID="cri-o://2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01" gracePeriod=30 Sep 15 07:46:14.854087 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:14.854062 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx"] Sep 15 07:46:14.854730 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:14.854329 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" podUID="86c8f32e-f6d9-4996-aa69-639be20202f3" containerName="payload-pre-processing" containerID="cri-o://af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e" gracePeriod=30 Sep 15 07:46:14.893672 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:14.893638 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:14.956681 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:14.956648 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:15.052626 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.052597 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:15.112994 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.112965 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" Sep 15 07:46:15.125750 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.125725 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:15.148773 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.148748 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" Sep 15 07:46:15.213271 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.213245 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wcc2\" (UniqueName: \"kubernetes.io/projected/03157279-7fb5-48b9-9fbf-06ee72000524-kube-api-access-9wcc2\") pod \"03157279-7fb5-48b9-9fbf-06ee72000524\" (UID: \"03157279-7fb5-48b9-9fbf-06ee72000524\") " Sep 15 07:46:15.213417 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.213281 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03157279-7fb5-48b9-9fbf-06ee72000524-plugins-config-volume\") pod \"03157279-7fb5-48b9-9fbf-06ee72000524\" (UID: \"03157279-7fb5-48b9-9fbf-06ee72000524\") " Sep 15 07:46:15.213417 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.213317 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn8bj\" (UniqueName: \"kubernetes.io/projected/86c8f32e-f6d9-4996-aa69-639be20202f3-kube-api-access-pn8bj\") pod \"86c8f32e-f6d9-4996-aa69-639be20202f3\" (UID: \"86c8f32e-f6d9-4996-aa69-639be20202f3\") " Sep 15 07:46:15.213417 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.213340 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86c8f32e-f6d9-4996-aa69-639be20202f3-plugins-config-volume\") pod \"86c8f32e-f6d9-4996-aa69-639be20202f3\" (UID: \"86c8f32e-f6d9-4996-aa69-639be20202f3\") " Sep 15 07:46:15.213741 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.213705 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03157279-7fb5-48b9-9fbf-06ee72000524-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "03157279-7fb5-48b9-9fbf-06ee72000524" (UID: "03157279-7fb5-48b9-9fbf-06ee72000524"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:46:15.213854 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.213717 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86c8f32e-f6d9-4996-aa69-639be20202f3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "86c8f32e-f6d9-4996-aa69-639be20202f3" (UID: "86c8f32e-f6d9-4996-aa69-639be20202f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:46:15.215402 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.215381 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03157279-7fb5-48b9-9fbf-06ee72000524-kube-api-access-9wcc2" (OuterVolumeSpecName: "kube-api-access-9wcc2") pod "03157279-7fb5-48b9-9fbf-06ee72000524" (UID: "03157279-7fb5-48b9-9fbf-06ee72000524"). InnerVolumeSpecName "kube-api-access-9wcc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:46:15.215478 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.215413 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86c8f32e-f6d9-4996-aa69-639be20202f3-kube-api-access-pn8bj" (OuterVolumeSpecName: "kube-api-access-pn8bj") pod "86c8f32e-f6d9-4996-aa69-639be20202f3" (UID: "86c8f32e-f6d9-4996-aa69-639be20202f3"). InnerVolumeSpecName "kube-api-access-pn8bj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:46:15.307429 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.307394 2552 generic.go:356] "Generic (PLEG): container finished" podID="03157279-7fb5-48b9-9fbf-06ee72000524" containerID="2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01" exitCode=0 Sep 15 07:46:15.307619 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.307445 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" Sep 15 07:46:15.307619 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.307477 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" event={"ID":"03157279-7fb5-48b9-9fbf-06ee72000524","Type":"ContainerDied","Data":"2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01"} Sep 15 07:46:15.307619 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.307558 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn" event={"ID":"03157279-7fb5-48b9-9fbf-06ee72000524","Type":"ContainerDied","Data":"fd1b7247f47594abd9242ea54fb421584039e06ca2e190ab5e20765d517154e0"} Sep 15 07:46:15.307619 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.307591 2552 scope.go:117] "RemoveContainer" containerID="2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01" Sep 15 07:46:15.308978 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.308957 2552 generic.go:356] "Generic (PLEG): container finished" podID="86c8f32e-f6d9-4996-aa69-639be20202f3" containerID="af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e" exitCode=0 Sep 15 07:46:15.309095 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.309005 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" Sep 15 07:46:15.309095 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.309040 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" event={"ID":"86c8f32e-f6d9-4996-aa69-639be20202f3","Type":"ContainerDied","Data":"af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e"} Sep 15 07:46:15.309095 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.309089 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx" event={"ID":"86c8f32e-f6d9-4996-aa69-639be20202f3","Type":"ContainerDied","Data":"41dc86189372aeb8b514307bda791934e5b6a2a8d768ab98f7c0358ca8d5caf7"} Sep 15 07:46:15.314305 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.314274 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wcc2\" (UniqueName: \"kubernetes.io/projected/03157279-7fb5-48b9-9fbf-06ee72000524-kube-api-access-9wcc2\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:46:15.314305 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.314297 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03157279-7fb5-48b9-9fbf-06ee72000524-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:46:15.314305 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.314306 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn8bj\" (UniqueName: \"kubernetes.io/projected/86c8f32e-f6d9-4996-aa69-639be20202f3-kube-api-access-pn8bj\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:46:15.314467 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.314315 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86c8f32e-f6d9-4996-aa69-639be20202f3-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:46:15.320735 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.320639 2552 scope.go:117] "RemoveContainer" containerID="2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01" Sep 15 07:46:15.321052 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:46:15.321025 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01\": container with ID starting with 2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01 not found: ID does not exist" containerID="2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01" Sep 15 07:46:15.321151 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.321058 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01"} err="failed to get container status \"2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01\": rpc error: code = NotFound desc = could not find container \"2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01\": container with ID starting with 2b9e1f043e1999f7194d126d5c8f94237f2842dc2b689407b3bee78dc1926a01 not found: ID does not exist" Sep 15 07:46:15.321151 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.321079 2552 scope.go:117] "RemoveContainer" containerID="af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e" Sep 15 07:46:15.329348 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.329331 2552 scope.go:117] "RemoveContainer" containerID="af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e" Sep 15 07:46:15.329605 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:46:15.329587 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e\": container with ID starting with af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e not found: ID does not exist" containerID="af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e" Sep 15 07:46:15.329657 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.329610 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e"} err="failed to get container status \"af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e\": rpc error: code = NotFound desc = could not find container \"af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e\": container with ID starting with af115199c2bf7f7ec0dac590185f6420cb88a5c8511312582cf059f3855ccf5e not found: ID does not exist" Sep 15 07:46:15.336850 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.336830 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx"] Sep 15 07:46:15.339050 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.338994 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b21d1ff2-d74bbc754-mhtsx"] Sep 15 07:46:15.347972 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.347954 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn"] Sep 15 07:46:15.351362 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.351344 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b21d1ff2-64d6fcc577-lhlqn"] Sep 15 07:46:15.474692 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.474664 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03157279-7fb5-48b9-9fbf-06ee72000524" path="/var/lib/kubelet/pods/03157279-7fb5-48b9-9fbf-06ee72000524/volumes" Sep 15 07:46:15.475000 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.474987 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5898b14d-d6b2-4fd5-b0fb-1557192cab80" path="/var/lib/kubelet/pods/5898b14d-d6b2-4fd5-b0fb-1557192cab80/volumes" Sep 15 07:46:15.475278 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.475267 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86c8f32e-f6d9-4996-aa69-639be20202f3" path="/var/lib/kubelet/pods/86c8f32e-f6d9-4996-aa69-639be20202f3/volumes" Sep 15 07:46:15.639796 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:15.639709 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:16.334767 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:16.334737 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:17.252917 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:17.252861 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:18.030302 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:18.030267 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:18.126058 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:18.126025 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:20.069013 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:20.068982 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:20.169776 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:20.169746 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:20.271627 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:20.271589 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:23.106322 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:23.106285 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:24.809612 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:24.809581 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:31.761330 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:31.761294 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:33.168494 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:33.168461 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:42.528256 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:42.528222 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:51.306457 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:51.306420 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:58.739687 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:58.739647 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:46:59.956555 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:46:59.956522 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:00.250612 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:00.250583 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:04.798066 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:04.798032 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:05.975516 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:05.975489 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:11.553399 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:11.553365 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:11.621414 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:11.621376 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:11.787396 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:11.787357 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:11.886843 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:11.886749 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:12.981068 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:12.981033 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:13.081048 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:13.081018 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:13.183615 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:13.183576 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:13.782142 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:13.782115 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:13.875157 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:13.875103 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:15.180439 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:15.180380 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:20.875923 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:20.875884 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:20.975983 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:20.975952 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:21.523696 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:21.523664 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:27.705048 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:27.705007 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:27.764909 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:27.764860 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:27.843406 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:27.843373 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:27.942487 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:27.942457 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:38.367495 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:38.367464 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:38.583806 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:38.583773 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:39.651311 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:39.651283 2552 scope.go:117] "RemoveContainer" containerID="b29141a0c9530bf790b6cfc804549d07cae2c83119f76cf2248c3a928447f6b7" Sep 15 07:47:41.722227 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:41.722191 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:47:59.476200 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:47:59.476164 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:03.992693 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:03.992658 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:04.057825 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:04.057790 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:05.415101 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:05.415071 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:05.515301 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:05.515268 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:05.748815 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:05.748785 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:06.041767 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:06.041680 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:06.192802 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:06.192765 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:09.475448 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.475410 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:09.596814 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.596780 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn"] Sep 15 07:48:09.597230 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597216 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e2be7f1-b12f-4f1d-9b97-086f223850ee" containerName="cleanup" Sep 15 07:48:09.597230 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597231 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e2be7f1-b12f-4f1d-9b97-086f223850ee" containerName="cleanup" Sep 15 07:48:09.597314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597244 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a4a587-309f-4e9c-91df-5137e35b3f91" containerName="cleanup" Sep 15 07:48:09.597314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597249 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a4a587-309f-4e9c-91df-5137e35b3f91" containerName="cleanup" Sep 15 07:48:09.597314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597259 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5898b14d-d6b2-4fd5-b0fb-1557192cab80" containerName="cleanup" Sep 15 07:48:09.597314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597265 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5898b14d-d6b2-4fd5-b0fb-1557192cab80" containerName="cleanup" Sep 15 07:48:09.597314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597271 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03157279-7fb5-48b9-9fbf-06ee72000524" containerName="payload-processing" Sep 15 07:48:09.597314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597277 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="03157279-7fb5-48b9-9fbf-06ee72000524" containerName="payload-processing" Sep 15 07:48:09.597314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597283 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86c8f32e-f6d9-4996-aa69-639be20202f3" containerName="payload-pre-processing" Sep 15 07:48:09.597314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597288 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="86c8f32e-f6d9-4996-aa69-639be20202f3" containerName="payload-pre-processing" Sep 15 07:48:09.597314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597296 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5b8698-3bac-43f6-8428-be56b15d8405" containerName="cleanup" Sep 15 07:48:09.597314 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597301 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5b8698-3bac-43f6-8428-be56b15d8405" containerName="cleanup" Sep 15 07:48:09.597608 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597358 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="03157279-7fb5-48b9-9fbf-06ee72000524" containerName="payload-processing" Sep 15 07:48:09.597608 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597365 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e5b8698-3bac-43f6-8428-be56b15d8405" containerName="cleanup" Sep 15 07:48:09.597608 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597371 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="86c8f32e-f6d9-4996-aa69-639be20202f3" containerName="payload-pre-processing" Sep 15 07:48:09.597608 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597376 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e2be7f1-b12f-4f1d-9b97-086f223850ee" containerName="cleanup" Sep 15 07:48:09.597608 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597384 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5898b14d-d6b2-4fd5-b0fb-1557192cab80" containerName="cleanup" Sep 15 07:48:09.597608 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.597389 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a4a587-309f-4e9c-91df-5137e35b3f91" containerName="cleanup" Sep 15 07:48:09.600655 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.600632 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn" Sep 15 07:48:09.603002 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.602977 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8jpwk\"" Sep 15 07:48:09.603544 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.602999 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 07:48:09.607248 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.607227 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn"] Sep 15 07:48:09.727810 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.727720 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:09.745379 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.745342 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc1cc43-9c98-4e06-a2ef-d54a72f195db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn\" (UID: \"0bc1cc43-9c98-4e06-a2ef-d54a72f195db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn" Sep 15 07:48:09.839970 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.838055 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:09.846422 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.846379 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc1cc43-9c98-4e06-a2ef-d54a72f195db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn\" (UID: \"0bc1cc43-9c98-4e06-a2ef-d54a72f195db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn" Sep 15 07:48:09.847262 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.847238 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc1cc43-9c98-4e06-a2ef-d54a72f195db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn\" (UID: \"0bc1cc43-9c98-4e06-a2ef-d54a72f195db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn" Sep 15 07:48:09.915369 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:09.915336 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn" Sep 15 07:48:10.188624 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:10.188601 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn"] Sep 15 07:48:10.190487 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:48:10.190442 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bc1cc43_9c98_4e06_a2ef_d54a72f195db.slice/crio-8925d9a1ff8f848a75120a1f1a186525ec32605009f7e820d3eb1d86e2180d44 WatchSource:0}: Error finding container 8925d9a1ff8f848a75120a1f1a186525ec32605009f7e820d3eb1d86e2180d44: Status 404 returned error can't find the container with id 8925d9a1ff8f848a75120a1f1a186525ec32605009f7e820d3eb1d86e2180d44 Sep 15 07:48:10.534773 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:10.534741 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:10.950762 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:10.950722 2552 generic.go:356] "Generic (PLEG): container finished" podID="0bc1cc43-9c98-4e06-a2ef-d54a72f195db" containerID="a815cfeb29412fd3fe81c77e5a40fdf9326686e2f028b37fd3a3774c33d25909" exitCode=0 Sep 15 07:48:10.950951 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:10.950767 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn" event={"ID":"0bc1cc43-9c98-4e06-a2ef-d54a72f195db","Type":"ContainerDied","Data":"a815cfeb29412fd3fe81c77e5a40fdf9326686e2f028b37fd3a3774c33d25909"} Sep 15 07:48:10.950951 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:10.950835 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn" event={"ID":"0bc1cc43-9c98-4e06-a2ef-d54a72f195db","Type":"ContainerStarted","Data":"8925d9a1ff8f848a75120a1f1a186525ec32605009f7e820d3eb1d86e2180d44"} Sep 15 07:48:11.379990 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:11.379958 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:12.098914 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.098891 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn" Sep 15 07:48:12.113323 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.113298 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck"] Sep 15 07:48:12.113758 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.113741 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bc1cc43-9c98-4e06-a2ef-d54a72f195db" containerName="revoke-keys" Sep 15 07:48:12.113831 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.113761 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bc1cc43-9c98-4e06-a2ef-d54a72f195db" containerName="revoke-keys" Sep 15 07:48:12.113904 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.113887 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bc1cc43-9c98-4e06-a2ef-d54a72f195db" containerName="revoke-keys" Sep 15 07:48:12.116922 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.116904 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.119499 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.119478 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f56165\"/\"distinct-2-w4-f56165-kserve-self-signed-certs\"" Sep 15 07:48:12.119662 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.119487 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-f56165\"/\"default-dockercfg-p295w\"" Sep 15 07:48:12.119782 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.119539 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-f56165\"/\"kube-root-ca.crt\"" Sep 15 07:48:12.120260 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.120238 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-f56165\"/\"openshift-service-ca.crt\"" Sep 15 07:48:12.126488 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.126468 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck"] Sep 15 07:48:12.172564 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.172533 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc1cc43-9c98-4e06-a2ef-d54a72f195db-maas-api-service-ca\") pod \"0bc1cc43-9c98-4e06-a2ef-d54a72f195db\" (UID: \"0bc1cc43-9c98-4e06-a2ef-d54a72f195db\") " Sep 15 07:48:12.172881 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.172846 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bc1cc43-9c98-4e06-a2ef-d54a72f195db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0bc1cc43-9c98-4e06-a2ef-d54a72f195db" (UID: "0bc1cc43-9c98-4e06-a2ef-d54a72f195db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:48:12.173039 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.172978 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc1cc43-9c98-4e06-a2ef-d54a72f195db-maas-api-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:48:12.274431 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.274399 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-tmp-dir\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.274431 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.274434 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-home\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.274633 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.274451 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-model-cache\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.274633 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.274572 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3852c36-8497-4de4-bbef-717d8d7cd96d-tls-certs\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.274633 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.274619 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pmnk\" (UniqueName: \"kubernetes.io/projected/f3852c36-8497-4de4-bbef-717d8d7cd96d-kube-api-access-2pmnk\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.274741 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.274667 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-dshm\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.375622 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.375586 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-tmp-dir\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.375622 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.375627 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-home\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.375906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.375644 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-model-cache\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.375906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.375699 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3852c36-8497-4de4-bbef-717d8d7cd96d-tls-certs\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.375906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.375727 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pmnk\" (UniqueName: \"kubernetes.io/projected/f3852c36-8497-4de4-bbef-717d8d7cd96d-kube-api-access-2pmnk\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.375906 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.375766 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-dshm\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.376101 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.376074 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-tmp-dir\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.376146 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.376130 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-home\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.376183 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.376153 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-model-cache\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.378003 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.377975 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-dshm\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.378218 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.378198 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3852c36-8497-4de4-bbef-717d8d7cd96d-tls-certs\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.384056 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.384030 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pmnk\" (UniqueName: \"kubernetes.io/projected/f3852c36-8497-4de4-bbef-717d8d7cd96d-kube-api-access-2pmnk\") pod \"distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.430726 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.430705 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:12.721148 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.721124 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck"] Sep 15 07:48:12.723045 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:48:12.723021 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3852c36_8497_4de4_bbef_717d8d7cd96d.slice/crio-a5ebdf4622e3742759fe542636581bf60d392fc1a040ddee682c287d9308481a WatchSource:0}: Error finding container a5ebdf4622e3742759fe542636581bf60d392fc1a040ddee682c287d9308481a: Status 404 returned error can't find the container with id a5ebdf4622e3742759fe542636581bf60d392fc1a040ddee682c287d9308481a Sep 15 07:48:12.962840 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.962809 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn" Sep 15 07:48:12.963037 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.962848 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn" event={"ID":"0bc1cc43-9c98-4e06-a2ef-d54a72f195db","Type":"ContainerDied","Data":"8925d9a1ff8f848a75120a1f1a186525ec32605009f7e820d3eb1d86e2180d44"} Sep 15 07:48:12.963037 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.962911 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8925d9a1ff8f848a75120a1f1a186525ec32605009f7e820d3eb1d86e2180d44" Sep 15 07:48:12.964495 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.964456 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" event={"ID":"f3852c36-8497-4de4-bbef-717d8d7cd96d","Type":"ContainerStarted","Data":"c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094"} Sep 15 07:48:12.964495 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.964496 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" event={"ID":"f3852c36-8497-4de4-bbef-717d8d7cd96d","Type":"ContainerStarted","Data":"a5ebdf4622e3742759fe542636581bf60d392fc1a040ddee682c287d9308481a"} Sep 15 07:48:12.983227 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:12.983184 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" podStartSLOduration=0.983170648 podStartE2EDuration="983.170648ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:48:12.981365851 +0000 UTC m=+1773.962528017" watchObservedRunningTime="2026-09-15 07:48:12.983170648 +0000 UTC m=+1773.964332862" Sep 15 07:48:15.492826 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.492795 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq"] Sep 15 07:48:15.496811 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.496789 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" Sep 15 07:48:15.499653 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.499634 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-4ef86b-dockercfg-k57ts\"" Sep 15 07:48:15.500056 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.500039 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-4ef86b\"" Sep 15 07:48:15.515769 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.515744 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq"] Sep 15 07:48:15.614944 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.614902 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq\" (UID: \"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" Sep 15 07:48:15.615616 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.615220 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpg8s\" (UniqueName: \"kubernetes.io/projected/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03-kube-api-access-hpg8s\") pod \"payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq\" (UID: \"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" Sep 15 07:48:15.647077 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.647035 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc"] Sep 15 07:48:15.650980 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.650959 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" Sep 15 07:48:15.661216 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.661186 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc"] Sep 15 07:48:15.669234 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.669210 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:15.716997 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.716963 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq\" (UID: \"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" Sep 15 07:48:15.717163 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.717004 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpg8s\" (UniqueName: \"kubernetes.io/projected/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03-kube-api-access-hpg8s\") pod \"payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq\" (UID: \"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" Sep 15 07:48:15.717703 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.717679 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq\" (UID: \"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" Sep 15 07:48:15.726461 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.726431 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpg8s\" (UniqueName: \"kubernetes.io/projected/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03-kube-api-access-hpg8s\") pod \"payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq\" (UID: \"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" Sep 15 07:48:15.808196 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.808107 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" Sep 15 07:48:15.818284 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.818261 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7l2r\" (UniqueName: \"kubernetes.io/projected/f49576f9-c00d-458f-9daa-59381aa77ca6-kube-api-access-v7l2r\") pod \"payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc\" (UID: \"f49576f9-c00d-458f-9daa-59381aa77ca6\") " pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" Sep 15 07:48:15.818378 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.818298 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f49576f9-c00d-458f-9daa-59381aa77ca6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc\" (UID: \"f49576f9-c00d-458f-9daa-59381aa77ca6\") " pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" Sep 15 07:48:15.919233 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.919199 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7l2r\" (UniqueName: \"kubernetes.io/projected/f49576f9-c00d-458f-9daa-59381aa77ca6-kube-api-access-v7l2r\") pod \"payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc\" (UID: \"f49576f9-c00d-458f-9daa-59381aa77ca6\") " pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" Sep 15 07:48:15.919409 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.919250 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f49576f9-c00d-458f-9daa-59381aa77ca6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc\" (UID: \"f49576f9-c00d-458f-9daa-59381aa77ca6\") " pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" Sep 15 07:48:15.919916 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.919899 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f49576f9-c00d-458f-9daa-59381aa77ca6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc\" (UID: \"f49576f9-c00d-458f-9daa-59381aa77ca6\") " pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" Sep 15 07:48:15.930979 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.930956 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7l2r\" (UniqueName: \"kubernetes.io/projected/f49576f9-c00d-458f-9daa-59381aa77ca6-kube-api-access-v7l2r\") pod \"payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc\" (UID: \"f49576f9-c00d-458f-9daa-59381aa77ca6\") " pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" Sep 15 07:48:15.952652 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.952616 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:15.962561 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:15.962544 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" Sep 15 07:48:16.083605 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:16.083579 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq"] Sep 15 07:48:16.084801 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:48:16.084777 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cbfaaa0_da3c_446e_b078_cbc01cf9fe03.slice/crio-a0e2ea945b0e9e1cf6cbc438103034c334b6dbc16c5f381b98fd67fb1d0c5d9b WatchSource:0}: Error finding container a0e2ea945b0e9e1cf6cbc438103034c334b6dbc16c5f381b98fd67fb1d0c5d9b: Status 404 returned error can't find the container with id a0e2ea945b0e9e1cf6cbc438103034c334b6dbc16c5f381b98fd67fb1d0c5d9b Sep 15 07:48:16.258359 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:16.258328 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc"] Sep 15 07:48:16.683751 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:16.683717 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:16.991283 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:16.991251 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" event={"ID":"f49576f9-c00d-458f-9daa-59381aa77ca6","Type":"ContainerStarted","Data":"2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253"} Sep 15 07:48:16.991464 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:16.991327 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" Sep 15 07:48:16.991464 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:16.991351 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" event={"ID":"f49576f9-c00d-458f-9daa-59381aa77ca6","Type":"ContainerStarted","Data":"e02e1bca6d4c0f336221e18f3c9132206e79867f08c4f950b478b4461fff1267"} Sep 15 07:48:16.992973 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:16.992942 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" event={"ID":"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03","Type":"ContainerStarted","Data":"c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed"} Sep 15 07:48:16.993087 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:16.992997 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" event={"ID":"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03","Type":"ContainerStarted","Data":"a0e2ea945b0e9e1cf6cbc438103034c334b6dbc16c5f381b98fd67fb1d0c5d9b"} Sep 15 07:48:16.993087 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:16.993058 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" Sep 15 07:48:17.013992 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:17.013936 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" podStartSLOduration=2.013917419 podStartE2EDuration="2.013917419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:48:17.010774626 +0000 UTC m=+1777.991936796" watchObservedRunningTime="2026-09-15 07:48:17.013917419 +0000 UTC m=+1777.995079590" Sep 15 07:48:17.030012 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:17.029966 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" podStartSLOduration=2.029947762 podStartE2EDuration="2.029947762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:48:17.02928124 +0000 UTC m=+1778.010443408" watchObservedRunningTime="2026-09-15 07:48:17.029947762 +0000 UTC m=+1778.011109931" Sep 15 07:48:17.561514 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:17.561481 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:19.794589 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:19.794555 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:22.431542 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:22.431504 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:22.432086 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:22.431556 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:22.444596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:22.444571 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:23.050423 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:23.050395 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:48:25.655297 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:25.655261 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:25.725160 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:25.725124 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj"] Sep 15 07:48:25.725434 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:25.725401 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" podUID="f788e9db-15b8-4741-bd21-1c642563e556" containerName="payload-processing" containerID="cri-o://1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a" gracePeriod=30 Sep 15 07:48:25.739453 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:25.739414 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:25.808561 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:25.808528 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:25.908107 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:25.908021 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:25.924266 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:25.924240 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn"] Sep 15 07:48:25.927990 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:25.927964 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2735c5-jpvnn"] Sep 15 07:48:25.983717 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:25.983695 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" Sep 15 07:48:26.012581 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.012533 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:26.015911 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.015857 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4j58z\" (UniqueName: \"kubernetes.io/projected/f788e9db-15b8-4741-bd21-1c642563e556-kube-api-access-4j58z\") pod \"f788e9db-15b8-4741-bd21-1c642563e556\" (UID: \"f788e9db-15b8-4741-bd21-1c642563e556\") " Sep 15 07:48:26.016050 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.016002 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f788e9db-15b8-4741-bd21-1c642563e556-plugins-config-volume\") pod \"f788e9db-15b8-4741-bd21-1c642563e556\" (UID: \"f788e9db-15b8-4741-bd21-1c642563e556\") " Sep 15 07:48:26.016389 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.016358 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f788e9db-15b8-4741-bd21-1c642563e556-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f788e9db-15b8-4741-bd21-1c642563e556" (UID: "f788e9db-15b8-4741-bd21-1c642563e556"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:48:26.018258 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.018230 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f788e9db-15b8-4741-bd21-1c642563e556-kube-api-access-4j58z" (OuterVolumeSpecName: "kube-api-access-4j58z") pod "f788e9db-15b8-4741-bd21-1c642563e556" (UID: "f788e9db-15b8-4741-bd21-1c642563e556"). InnerVolumeSpecName "kube-api-access-4j58z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:48:26.054760 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.054731 2552 generic.go:356] "Generic (PLEG): container finished" podID="f788e9db-15b8-4741-bd21-1c642563e556" containerID="1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a" exitCode=0 Sep 15 07:48:26.054905 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.054788 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" event={"ID":"f788e9db-15b8-4741-bd21-1c642563e556","Type":"ContainerDied","Data":"1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a"} Sep 15 07:48:26.054905 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.054791 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" Sep 15 07:48:26.054905 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.054887 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj" event={"ID":"f788e9db-15b8-4741-bd21-1c642563e556","Type":"ContainerDied","Data":"be309fddc1737dbd9b5b65b084d1d667bced446fec6ad61a48fb156bbbd0fe7c"} Sep 15 07:48:26.054905 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.054893 2552 scope.go:117] "RemoveContainer" containerID="1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a" Sep 15 07:48:26.065118 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.065100 2552 scope.go:117] "RemoveContainer" containerID="1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a" Sep 15 07:48:26.065350 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:48:26.065333 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a\": container with ID starting with 1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a not found: ID does not exist" containerID="1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a" Sep 15 07:48:26.065399 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.065356 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a"} err="failed to get container status \"1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a\": rpc error: code = NotFound desc = could not find container \"1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a\": container with ID starting with 1f51e4b1ae13dc0378f1f105163bf8373d9c71b5aa15f0678b7febeee7d0045a not found: ID does not exist" Sep 15 07:48:26.080375 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.080348 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj"] Sep 15 07:48:26.086200 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.086176 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2735c5-fdc8c9bcc-q6jfj"] Sep 15 07:48:26.117353 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.117326 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4j58z\" (UniqueName: \"kubernetes.io/projected/f788e9db-15b8-4741-bd21-1c642563e556-kube-api-access-4j58z\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:48:26.117353 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.117348 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f788e9db-15b8-4741-bd21-1c642563e556-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:48:26.392604 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.392560 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:26.497932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:26.497893 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:27.365563 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:27.365533 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:27.475439 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:27.475407 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bc1cc43-9c98-4e06-a2ef-d54a72f195db" path="/var/lib/kubelet/pods/0bc1cc43-9c98-4e06-a2ef-d54a72f195db/volumes" Sep 15 07:48:27.475912 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:27.475891 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f788e9db-15b8-4741-bd21-1c642563e556" path="/var/lib/kubelet/pods/f788e9db-15b8-4741-bd21-1c642563e556/volumes" Sep 15 07:48:28.001911 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:28.001879 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" Sep 15 07:48:28.002082 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:28.002036 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" Sep 15 07:48:28.254648 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:28.254560 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:29.048073 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:29.048033 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:30.038321 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:30.038279 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:31.097984 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:31.097952 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:32.152680 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:32.152601 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:32.783439 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:32.783405 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:32.890919 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:32.890862 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:33.047760 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:33.047680 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q"] Sep 15 07:48:33.047981 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:33.047951 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerName="istio-proxy" containerID="cri-o://90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f" gracePeriod=30 Sep 15 07:48:33.074956 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:33.074915 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:33.675688 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:33.675651 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:34.583520 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:34.583482 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:34.702361 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:34.702323 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:35.040174 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:35.040139 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:35.334083 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:35.333996 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:35.436394 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:35.436363 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:36.077107 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:36.077063 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:36.537115 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:36.537084 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:37.565248 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:37.565211 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:38.313913 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.313889 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:48:38.441639 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.441549 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istiod-ca-cert\") pod \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " Sep 15 07:48:38.441639 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.441627 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-workload-certs\") pod \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " Sep 15 07:48:38.441897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.441674 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-data\") pod \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " Sep 15 07:48:38.441897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.441714 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-workload-socket\") pod \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " Sep 15 07:48:38.441897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.441767 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rx6b\" (UniqueName: \"kubernetes.io/projected/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-kube-api-access-4rx6b\") pod \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " Sep 15 07:48:38.441897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.441791 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-envoy\") pod \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " Sep 15 07:48:38.441897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.441820 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-token\") pod \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " Sep 15 07:48:38.442165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.441913 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-podinfo\") pod \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " Sep 15 07:48:38.442165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.441925 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fefaa36f-1ec1-4baf-83ef-f612b58b6cea" (UID: "fefaa36f-1ec1-4baf-83ef-f612b58b6cea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:48:38.442165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.441962 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-credential-socket\") pod \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\" (UID: \"fefaa36f-1ec1-4baf-83ef-f612b58b6cea\") " Sep 15 07:48:38.442165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.442001 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fefaa36f-1ec1-4baf-83ef-f612b58b6cea" (UID: "fefaa36f-1ec1-4baf-83ef-f612b58b6cea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:48:38.442165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.442040 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fefaa36f-1ec1-4baf-83ef-f612b58b6cea" (UID: "fefaa36f-1ec1-4baf-83ef-f612b58b6cea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:48:38.442165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.442058 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-data" (OuterVolumeSpecName: "istio-data") pod "fefaa36f-1ec1-4baf-83ef-f612b58b6cea" (UID: "fefaa36f-1ec1-4baf-83ef-f612b58b6cea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:48:38.442487 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.442306 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fefaa36f-1ec1-4baf-83ef-f612b58b6cea" (UID: "fefaa36f-1ec1-4baf-83ef-f612b58b6cea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:48:38.442487 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.442407 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-workload-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:48:38.442487 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.442422 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-data\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:48:38.442487 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.442436 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-workload-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:48:38.442487 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.442450 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-credential-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:48:38.442487 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.442464 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istiod-ca-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:48:38.444091 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.444056 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-token" (OuterVolumeSpecName: "istio-token") pod "fefaa36f-1ec1-4baf-83ef-f612b58b6cea" (UID: "fefaa36f-1ec1-4baf-83ef-f612b58b6cea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:48:38.444173 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.444113 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fefaa36f-1ec1-4baf-83ef-f612b58b6cea" (UID: "fefaa36f-1ec1-4baf-83ef-f612b58b6cea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:48:38.444397 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.444379 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fefaa36f-1ec1-4baf-83ef-f612b58b6cea" (UID: "fefaa36f-1ec1-4baf-83ef-f612b58b6cea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 07:48:38.444453 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.444395 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-kube-api-access-4rx6b" (OuterVolumeSpecName: "kube-api-access-4rx6b") pod "fefaa36f-1ec1-4baf-83ef-f612b58b6cea" (UID: "fefaa36f-1ec1-4baf-83ef-f612b58b6cea"). InnerVolumeSpecName "kube-api-access-4rx6b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:48:38.543235 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.543204 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rx6b\" (UniqueName: \"kubernetes.io/projected/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-kube-api-access-4rx6b\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:48:38.543235 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.543234 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-envoy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:48:38.543235 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.543243 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-token\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:48:38.543487 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.543252 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fefaa36f-1ec1-4baf-83ef-f612b58b6cea-istio-podinfo\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:48:38.544261 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.544234 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:38.858493 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.858458 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:38.999961 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:38.999929 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:39.134416 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.134332 2552 generic.go:356] "Generic (PLEG): container finished" podID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerID="90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f" exitCode=0 Sep 15 07:48:39.134416 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.134365 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" event={"ID":"fefaa36f-1ec1-4baf-83ef-f612b58b6cea","Type":"ContainerDied","Data":"90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f"} Sep 15 07:48:39.134416 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.134385 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" Sep 15 07:48:39.134416 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.134405 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q" event={"ID":"fefaa36f-1ec1-4baf-83ef-f612b58b6cea","Type":"ContainerDied","Data":"3ddb4f624f677dc9f7af0c35764d035cabd21485fc06296b30e5742bfca48544"} Sep 15 07:48:39.134732 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.134426 2552 scope.go:117] "RemoveContainer" containerID="90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f" Sep 15 07:48:39.146593 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.146566 2552 scope.go:117] "RemoveContainer" containerID="76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0" Sep 15 07:48:39.155462 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.155440 2552 scope.go:117] "RemoveContainer" containerID="90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f" Sep 15 07:48:39.155703 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:48:39.155684 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f\": container with ID starting with 90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f not found: ID does not exist" containerID="90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f" Sep 15 07:48:39.155772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.155713 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f"} err="failed to get container status \"90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f\": rpc error: code = NotFound desc = could not find container \"90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f\": container with ID starting with 90dc24164da54d0089e667338128a9ed8ba4e013d5764256de65a701a7f78e3f not found: ID does not exist" Sep 15 07:48:39.155772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.155730 2552 scope.go:117] "RemoveContainer" containerID="76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0" Sep 15 07:48:39.156020 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:48:39.155999 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0\": container with ID starting with 76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0 not found: ID does not exist" containerID="76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0" Sep 15 07:48:39.156081 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.156026 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0"} err="failed to get container status \"76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0\": rpc error: code = NotFound desc = could not find container \"76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0\": container with ID starting with 76f140d15d42a0c4cf67ad2be12bcd573cca805c06eca3221f84e8cef8dbbfd0 not found: ID does not exist" Sep 15 07:48:39.158974 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.158954 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q"] Sep 15 07:48:39.167094 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.167075 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2735c5-openshift-default-6fbdd776db-xk48q"] Sep 15 07:48:39.384236 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.384208 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:39.478684 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:39.478651 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" path="/var/lib/kubelet/pods/fefaa36f-1ec1-4baf-83ef-f612b58b6cea/volumes" Sep 15 07:48:40.362057 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:40.362015 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:40.468752 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:40.468719 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:40.593435 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:40.593396 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:40.877919 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:40.877885 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:40.975794 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:40.975758 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:41.115466 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:41.115439 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:48:41.121050 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:41.121027 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:48:41.123980 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:41.123945 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:48:41.128979 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:41.128921 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:48:41.372640 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:41.372604 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:43.211942 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:43.211913 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:43.312811 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:43.312776 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:44.056329 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:44.056301 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:44.857369 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:44.857325 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:45.884383 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:45.884346 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:46.783325 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:46.783284 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:53.826749 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:53.826706 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:53.934932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:53.934841 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:48:54.214422 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:48:54.214336 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:00.061641 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:00.061604 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:01.420642 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:01.420597 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:02.837353 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:02.837305 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:03.653830 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:03.653786 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:03.798023 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:03.797990 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:04.505210 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:04.505178 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:05.550467 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:05.550435 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:06.349042 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:06.349006 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:06.488348 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:06.488322 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:07.354118 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:07.354087 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:08.338389 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:08.338349 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:09.190400 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:09.190358 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:10.089735 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:10.089694 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:11.039635 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:11.039601 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:12.882442 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:12.882398 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:13.150690 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:13.150599 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:14.074233 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:14.074194 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:16.974429 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:16.974383 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:20.457646 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:20.457611 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:22.772294 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:22.772251 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:22.795064 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:22.795034 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc"] Sep 15 07:49:22.795522 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:22.795489 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" podUID="f49576f9-c00d-458f-9daa-59381aa77ca6" containerName="payload-processing" containerID="cri-o://2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253" gracePeriod=30 Sep 15 07:49:22.808403 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:22.807160 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq"] Sep 15 07:49:22.808403 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:22.807383 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" podUID="6cbfaaa0-da3c-446e-b078-cbc01cf9fe03" containerName="payload-pre-processing" containerID="cri-o://c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed" gracePeriod=30 Sep 15 07:49:22.830046 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:49:22.829764 2552 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf49576f9_c00d_458f_9daa_59381aa77ca6.slice/crio-2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253.scope\": RecentStats: unable to find data in memory cache]" Sep 15 07:49:22.988391 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:22.988360 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:23.056100 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.055999 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:23.090742 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.090718 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" Sep 15 07:49:23.093617 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.093592 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" Sep 15 07:49:23.176684 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.176652 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f49576f9-c00d-458f-9daa-59381aa77ca6-plugins-config-volume\") pod \"f49576f9-c00d-458f-9daa-59381aa77ca6\" (UID: \"f49576f9-c00d-458f-9daa-59381aa77ca6\") " Sep 15 07:49:23.176932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.176692 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpg8s\" (UniqueName: \"kubernetes.io/projected/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03-kube-api-access-hpg8s\") pod \"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03\" (UID: \"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03\") " Sep 15 07:49:23.176932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.176735 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03-plugins-config-volume\") pod \"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03\" (UID: \"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03\") " Sep 15 07:49:23.177232 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.176934 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7l2r\" (UniqueName: \"kubernetes.io/projected/f49576f9-c00d-458f-9daa-59381aa77ca6-kube-api-access-v7l2r\") pod \"f49576f9-c00d-458f-9daa-59381aa77ca6\" (UID: \"f49576f9-c00d-458f-9daa-59381aa77ca6\") " Sep 15 07:49:23.177232 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.177115 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f49576f9-c00d-458f-9daa-59381aa77ca6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f49576f9-c00d-458f-9daa-59381aa77ca6" (UID: "f49576f9-c00d-458f-9daa-59381aa77ca6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:49:23.177232 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.177163 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6cbfaaa0-da3c-446e-b078-cbc01cf9fe03" (UID: "6cbfaaa0-da3c-446e-b078-cbc01cf9fe03"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:49:23.177390 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.177342 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:49:23.177390 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.177365 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f49576f9-c00d-458f-9daa-59381aa77ca6-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:49:23.181005 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.179758 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03-kube-api-access-hpg8s" (OuterVolumeSpecName: "kube-api-access-hpg8s") pod "6cbfaaa0-da3c-446e-b078-cbc01cf9fe03" (UID: "6cbfaaa0-da3c-446e-b078-cbc01cf9fe03"). InnerVolumeSpecName "kube-api-access-hpg8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:49:23.185580 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.185540 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f49576f9-c00d-458f-9daa-59381aa77ca6-kube-api-access-v7l2r" (OuterVolumeSpecName: "kube-api-access-v7l2r") pod "f49576f9-c00d-458f-9daa-59381aa77ca6" (UID: "f49576f9-c00d-458f-9daa-59381aa77ca6"). InnerVolumeSpecName "kube-api-access-v7l2r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:49:23.277894 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.277856 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7l2r\" (UniqueName: \"kubernetes.io/projected/f49576f9-c00d-458f-9daa-59381aa77ca6-kube-api-access-v7l2r\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:49:23.277894 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.277891 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpg8s\" (UniqueName: \"kubernetes.io/projected/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03-kube-api-access-hpg8s\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:49:23.369732 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.369637 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:23.382186 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.382148 2552 generic.go:356] "Generic (PLEG): container finished" podID="6cbfaaa0-da3c-446e-b078-cbc01cf9fe03" containerID="c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed" exitCode=0 Sep 15 07:49:23.382350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.382191 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" Sep 15 07:49:23.382350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.382234 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" event={"ID":"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03","Type":"ContainerDied","Data":"c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed"} Sep 15 07:49:23.382350 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.382321 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq" event={"ID":"6cbfaaa0-da3c-446e-b078-cbc01cf9fe03","Type":"ContainerDied","Data":"a0e2ea945b0e9e1cf6cbc438103034c334b6dbc16c5f381b98fd67fb1d0c5d9b"} Sep 15 07:49:23.382501 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.382354 2552 scope.go:117] "RemoveContainer" containerID="c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed" Sep 15 07:49:23.383772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.383740 2552 generic.go:356] "Generic (PLEG): container finished" podID="f49576f9-c00d-458f-9daa-59381aa77ca6" containerID="2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253" exitCode=0 Sep 15 07:49:23.383896 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.383776 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" event={"ID":"f49576f9-c00d-458f-9daa-59381aa77ca6","Type":"ContainerDied","Data":"2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253"} Sep 15 07:49:23.383896 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.383787 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" Sep 15 07:49:23.383896 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.383817 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc" event={"ID":"f49576f9-c00d-458f-9daa-59381aa77ca6","Type":"ContainerDied","Data":"e02e1bca6d4c0f336221e18f3c9132206e79867f08c4f950b478b4461fff1267"} Sep 15 07:49:23.393233 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.393217 2552 scope.go:117] "RemoveContainer" containerID="c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed" Sep 15 07:49:23.393505 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:49:23.393486 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed\": container with ID starting with c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed not found: ID does not exist" containerID="c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed" Sep 15 07:49:23.393570 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.393511 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed"} err="failed to get container status \"c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed\": rpc error: code = NotFound desc = could not find container \"c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed\": container with ID starting with c364c56291646b51deb0655fbfedd1da1230f518b14739fa346a2a2a49467bed not found: ID does not exist" Sep 15 07:49:23.393570 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.393530 2552 scope.go:117] "RemoveContainer" containerID="2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253" Sep 15 07:49:23.402229 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.402080 2552 scope.go:117] "RemoveContainer" containerID="2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253" Sep 15 07:49:23.402742 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:49:23.402718 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253\": container with ID starting with 2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253 not found: ID does not exist" containerID="2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253" Sep 15 07:49:23.402816 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.402747 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253"} err="failed to get container status \"2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253\": rpc error: code = NotFound desc = could not find container \"2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253\": container with ID starting with 2e4f81199ba9e4e2bda58df729275e6013b32e41a7cf1afe472af966c55ab253 not found: ID does not exist" Sep 15 07:49:23.404360 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.404339 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq"] Sep 15 07:49:23.410249 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.410225 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4ef86b-6684f6f8cd-wwvrq"] Sep 15 07:49:23.419645 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.419624 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc"] Sep 15 07:49:23.426849 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.426828 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4ef86b-6f9844996f-7w6vc"] Sep 15 07:49:23.434686 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.434660 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:23.479851 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.479813 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cbfaaa0-da3c-446e-b078-cbc01cf9fe03" path="/var/lib/kubelet/pods/6cbfaaa0-da3c-446e-b078-cbc01cf9fe03/volumes" Sep 15 07:49:23.480404 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:23.480377 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f49576f9-c00d-458f-9daa-59381aa77ca6" path="/var/lib/kubelet/pods/f49576f9-c00d-458f-9daa-59381aa77ca6/volumes" Sep 15 07:49:24.321033 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:24.320989 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:25.276259 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:25.276225 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:26.135934 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:26.135891 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:26.858047 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:26.858008 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:27.606621 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:27.606588 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:28.443521 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:28.443467 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:29.882217 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:29.882180 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:29.981954 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:29.981912 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:30.078950 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.078917 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:30.396360 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396322 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q"] Sep 15 07:49:30.396728 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396716 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f49576f9-c00d-458f-9daa-59381aa77ca6" containerName="payload-processing" Sep 15 07:49:30.396773 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396730 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f49576f9-c00d-458f-9daa-59381aa77ca6" containerName="payload-processing" Sep 15 07:49:30.396773 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396751 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerName="istio-proxy" Sep 15 07:49:30.396773 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396757 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerName="istio-proxy" Sep 15 07:49:30.396773 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396763 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cbfaaa0-da3c-446e-b078-cbc01cf9fe03" containerName="payload-pre-processing" Sep 15 07:49:30.396773 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396769 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cbfaaa0-da3c-446e-b078-cbc01cf9fe03" containerName="payload-pre-processing" Sep 15 07:49:30.396953 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396780 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f788e9db-15b8-4741-bd21-1c642563e556" containerName="payload-processing" Sep 15 07:49:30.396953 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396786 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f788e9db-15b8-4741-bd21-1c642563e556" containerName="payload-processing" Sep 15 07:49:30.396953 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396799 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerName="istio-proxy" Sep 15 07:49:30.396953 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396805 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerName="istio-proxy" Sep 15 07:49:30.396953 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396877 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f49576f9-c00d-458f-9daa-59381aa77ca6" containerName="payload-processing" Sep 15 07:49:30.396953 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396888 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f788e9db-15b8-4741-bd21-1c642563e556" containerName="payload-processing" Sep 15 07:49:30.396953 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396899 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerName="istio-proxy" Sep 15 07:49:30.396953 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396906 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="fefaa36f-1ec1-4baf-83ef-f612b58b6cea" containerName="istio-proxy" Sep 15 07:49:30.396953 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.396911 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cbfaaa0-da3c-446e-b078-cbc01cf9fe03" containerName="payload-pre-processing" Sep 15 07:49:30.401546 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.401529 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q" Sep 15 07:49:30.404006 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.403982 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8jpwk\"" Sep 15 07:49:30.404738 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.404716 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 07:49:30.407110 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.407087 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q"] Sep 15 07:49:30.555755 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.555718 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afcd7a31-4d4c-4fbf-bee7-ce1117a11918-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1755f7-9c77q\" (UID: \"afcd7a31-4d4c-4fbf-bee7-ce1117a11918\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q" Sep 15 07:49:30.657268 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.657177 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afcd7a31-4d4c-4fbf-bee7-ce1117a11918-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1755f7-9c77q\" (UID: \"afcd7a31-4d4c-4fbf-bee7-ce1117a11918\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q" Sep 15 07:49:30.657798 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.657777 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afcd7a31-4d4c-4fbf-bee7-ce1117a11918-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1755f7-9c77q\" (UID: \"afcd7a31-4d4c-4fbf-bee7-ce1117a11918\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q" Sep 15 07:49:30.715039 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.715002 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q" Sep 15 07:49:30.821608 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.821568 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:30.987534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:30.987504 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q"] Sep 15 07:49:30.989622 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:49:30.989596 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafcd7a31_4d4c_4fbf_bee7_ce1117a11918.slice/crio-945b9239bd93e7cc3caa274f4b951665027f26c2d961e279b8ab86fc141fc8c0 WatchSource:0}: Error finding container 945b9239bd93e7cc3caa274f4b951665027f26c2d961e279b8ab86fc141fc8c0: Status 404 returned error can't find the container with id 945b9239bd93e7cc3caa274f4b951665027f26c2d961e279b8ab86fc141fc8c0 Sep 15 07:49:31.443882 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:31.443773 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q" event={"ID":"afcd7a31-4d4c-4fbf-bee7-ce1117a11918","Type":"ContainerStarted","Data":"615c8137c36497896abc64b076c71197b377e31fd5d7ba8e96964d919c0c574c"} Sep 15 07:49:31.443882 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:31.443835 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q" event={"ID":"afcd7a31-4d4c-4fbf-bee7-ce1117a11918","Type":"ContainerStarted","Data":"945b9239bd93e7cc3caa274f4b951665027f26c2d961e279b8ab86fc141fc8c0"} Sep 15 07:49:31.458369 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:31.458316 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q" podStartSLOduration=1.458302596 podStartE2EDuration="1.458302596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:49:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:49:31.457310043 +0000 UTC m=+1852.438472212" watchObservedRunningTime="2026-09-15 07:49:31.458302596 +0000 UTC m=+1852.439464763" Sep 15 07:49:31.645788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:31.645746 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:32.450263 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:32.450232 2552 generic.go:356] "Generic (PLEG): container finished" podID="afcd7a31-4d4c-4fbf-bee7-ce1117a11918" containerID="615c8137c36497896abc64b076c71197b377e31fd5d7ba8e96964d919c0c574c" exitCode=0 Sep 15 07:49:32.450682 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:32.450316 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q" event={"ID":"afcd7a31-4d4c-4fbf-bee7-ce1117a11918","Type":"ContainerDied","Data":"615c8137c36497896abc64b076c71197b377e31fd5d7ba8e96964d919c0c574c"} Sep 15 07:49:32.538799 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:32.538767 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:33.235528 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:33.235497 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:33.585786 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:33.585760 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q" Sep 15 07:49:33.689227 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:33.689196 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afcd7a31-4d4c-4fbf-bee7-ce1117a11918-maas-api-service-ca\") pod \"afcd7a31-4d4c-4fbf-bee7-ce1117a11918\" (UID: \"afcd7a31-4d4c-4fbf-bee7-ce1117a11918\") " Sep 15 07:49:33.689557 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:33.689535 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afcd7a31-4d4c-4fbf-bee7-ce1117a11918-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "afcd7a31-4d4c-4fbf-bee7-ce1117a11918" (UID: "afcd7a31-4d4c-4fbf-bee7-ce1117a11918"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:49:33.790915 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:33.790812 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afcd7a31-4d4c-4fbf-bee7-ce1117a11918-maas-api-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:49:34.153663 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:34.153574 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:34.462129 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:34.462040 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q" Sep 15 07:49:34.462129 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:34.462081 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q" event={"ID":"afcd7a31-4d4c-4fbf-bee7-ce1117a11918","Type":"ContainerDied","Data":"945b9239bd93e7cc3caa274f4b951665027f26c2d961e279b8ab86fc141fc8c0"} Sep 15 07:49:34.462325 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:34.462140 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="945b9239bd93e7cc3caa274f4b951665027f26c2d961e279b8ab86fc141fc8c0" Sep 15 07:49:34.970924 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:34.970888 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:35.848604 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:35.848576 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:36.619531 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:36.619491 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:46.223004 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:46.222964 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:46.415131 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:46.415090 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:46.482816 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:46.482756 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q"] Sep 15 07:49:46.485455 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:46.485430 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1755f7-9c77q"] Sep 15 07:49:46.486165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:46.486134 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:46.551476 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:46.551444 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:46.838298 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:46.838211 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:46.945723 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:46.945688 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:47.474628 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:47.474599 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afcd7a31-4d4c-4fbf-bee7-ce1117a11918" path="/var/lib/kubelet/pods/afcd7a31-4d4c-4fbf-bee7-ce1117a11918/volumes" Sep 15 07:49:50.004633 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:50.004599 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:50.105622 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:50.105592 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:50.239914 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:50.239882 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:50.978248 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:50.978215 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:52.890602 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:52.890568 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:52.988559 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:52.988522 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:53.130317 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.130273 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:53.462544 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.462509 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr"] Sep 15 07:49:53.462961 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.462947 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afcd7a31-4d4c-4fbf-bee7-ce1117a11918" containerName="revoke-keys" Sep 15 07:49:53.463022 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.462962 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="afcd7a31-4d4c-4fbf-bee7-ce1117a11918" containerName="revoke-keys" Sep 15 07:49:53.463062 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.463031 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="afcd7a31-4d4c-4fbf-bee7-ce1117a11918" containerName="revoke-keys" Sep 15 07:49:53.466268 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.466252 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr" Sep 15 07:49:53.468574 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.468548 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 07:49:53.468710 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.468646 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8jpwk\"" Sep 15 07:49:53.478141 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.477791 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr"] Sep 15 07:49:53.587427 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.587394 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b08213a1-594b-4f37-84f8-d0b923248cd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr\" (UID: \"b08213a1-594b-4f37-84f8-d0b923248cd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr" Sep 15 07:49:53.688489 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.688456 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b08213a1-594b-4f37-84f8-d0b923248cd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr\" (UID: \"b08213a1-594b-4f37-84f8-d0b923248cd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr" Sep 15 07:49:53.689075 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.689056 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b08213a1-594b-4f37-84f8-d0b923248cd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr\" (UID: \"b08213a1-594b-4f37-84f8-d0b923248cd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr" Sep 15 07:49:53.754246 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.754217 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:53.779203 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:53.779182 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr" Sep 15 07:49:54.053071 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:54.052995 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr"] Sep 15 07:49:54.056577 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:49:54.056552 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb08213a1_594b_4f37_84f8_d0b923248cd4.slice/crio-ceddb7c2cbb1ae4dfa3fb4f55fca692c6bb03d32f51d47dcd8ec094b0eac20e4 WatchSource:0}: Error finding container ceddb7c2cbb1ae4dfa3fb4f55fca692c6bb03d32f51d47dcd8ec094b0eac20e4: Status 404 returned error can't find the container with id ceddb7c2cbb1ae4dfa3fb4f55fca692c6bb03d32f51d47dcd8ec094b0eac20e4 Sep 15 07:49:54.557250 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:54.557216 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:54.569439 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:54.569409 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr" event={"ID":"b08213a1-594b-4f37-84f8-d0b923248cd4","Type":"ContainerStarted","Data":"2463a77eefea2f4dc9dd075631dcad04c25daa5c00b922c1f52b3d4c78f38882"} Sep 15 07:49:54.569630 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:54.569459 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr" event={"ID":"b08213a1-594b-4f37-84f8-d0b923248cd4","Type":"ContainerStarted","Data":"ceddb7c2cbb1ae4dfa3fb4f55fca692c6bb03d32f51d47dcd8ec094b0eac20e4"} Sep 15 07:49:54.584744 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:54.584688 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr" podStartSLOduration=1.584672139 podStartE2EDuration="1.584672139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:49:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:49:54.582684943 +0000 UTC m=+1875.563847111" watchObservedRunningTime="2026-09-15 07:49:54.584672139 +0000 UTC m=+1875.565834304" Sep 15 07:49:55.458905 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:55.458860 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:55.576209 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:55.576175 2552 generic.go:356] "Generic (PLEG): container finished" podID="b08213a1-594b-4f37-84f8-d0b923248cd4" containerID="2463a77eefea2f4dc9dd075631dcad04c25daa5c00b922c1f52b3d4c78f38882" exitCode=0 Sep 15 07:49:55.576410 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:55.576264 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr" event={"ID":"b08213a1-594b-4f37-84f8-d0b923248cd4","Type":"ContainerDied","Data":"2463a77eefea2f4dc9dd075631dcad04c25daa5c00b922c1f52b3d4c78f38882"} Sep 15 07:49:56.238539 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:56.238500 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:56.716898 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:56.716876 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr" Sep 15 07:49:56.817881 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:56.817834 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b08213a1-594b-4f37-84f8-d0b923248cd4-maas-api-service-ca\") pod \"b08213a1-594b-4f37-84f8-d0b923248cd4\" (UID: \"b08213a1-594b-4f37-84f8-d0b923248cd4\") " Sep 15 07:49:56.818180 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:56.818156 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b08213a1-594b-4f37-84f8-d0b923248cd4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b08213a1-594b-4f37-84f8-d0b923248cd4" (UID: "b08213a1-594b-4f37-84f8-d0b923248cd4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:49:56.918991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:56.918910 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b08213a1-594b-4f37-84f8-d0b923248cd4-maas-api-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:49:56.930732 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:56.930706 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:57.587622 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:57.587595 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr" Sep 15 07:49:57.587622 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:57.587614 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr" event={"ID":"b08213a1-594b-4f37-84f8-d0b923248cd4","Type":"ContainerDied","Data":"ceddb7c2cbb1ae4dfa3fb4f55fca692c6bb03d32f51d47dcd8ec094b0eac20e4"} Sep 15 07:49:57.587826 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:57.587656 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ceddb7c2cbb1ae4dfa3fb4f55fca692c6bb03d32f51d47dcd8ec094b0eac20e4" Sep 15 07:49:57.868699 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:57.868614 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:49:58.459049 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:49:58.459014 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:04.088988 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.088950 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:04.115023 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.114991 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk"] Sep 15 07:50:04.115210 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.115190 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" podUID="8506cc5b-22f3-48b0-be74-de711496d948" containerName="payload-processing" containerID="cri-o://86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f" gracePeriod=30 Sep 15 07:50:04.139788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.139753 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh"] Sep 15 07:50:04.140293 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.140262 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" podUID="3355ae2f-9b9b-46ef-8212-651aa8884535" containerName="payload-pre-processing" containerID="cri-o://0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777" gracePeriod=30 Sep 15 07:50:04.181452 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.181416 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:04.280191 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.280146 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:04.302369 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.302300 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr"] Sep 15 07:50:04.304556 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.304527 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a4c9f-xshvr"] Sep 15 07:50:04.357937 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.357906 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:04.426546 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.426522 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" Sep 15 07:50:04.429292 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.429273 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" Sep 15 07:50:04.494819 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.494013 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3355ae2f-9b9b-46ef-8212-651aa8884535-plugins-config-volume\") pod \"3355ae2f-9b9b-46ef-8212-651aa8884535\" (UID: \"3355ae2f-9b9b-46ef-8212-651aa8884535\") " Sep 15 07:50:04.494819 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.494096 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm462\" (UniqueName: \"kubernetes.io/projected/8506cc5b-22f3-48b0-be74-de711496d948-kube-api-access-sm462\") pod \"8506cc5b-22f3-48b0-be74-de711496d948\" (UID: \"8506cc5b-22f3-48b0-be74-de711496d948\") " Sep 15 07:50:04.494819 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.494172 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8506cc5b-22f3-48b0-be74-de711496d948-plugins-config-volume\") pod \"8506cc5b-22f3-48b0-be74-de711496d948\" (UID: \"8506cc5b-22f3-48b0-be74-de711496d948\") " Sep 15 07:50:04.494819 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.494200 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56dlc\" (UniqueName: \"kubernetes.io/projected/3355ae2f-9b9b-46ef-8212-651aa8884535-kube-api-access-56dlc\") pod \"3355ae2f-9b9b-46ef-8212-651aa8884535\" (UID: \"3355ae2f-9b9b-46ef-8212-651aa8884535\") " Sep 15 07:50:04.494819 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.494448 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3355ae2f-9b9b-46ef-8212-651aa8884535-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3355ae2f-9b9b-46ef-8212-651aa8884535" (UID: "3355ae2f-9b9b-46ef-8212-651aa8884535"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:50:04.494819 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.494642 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3355ae2f-9b9b-46ef-8212-651aa8884535-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:04.495451 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.495420 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8506cc5b-22f3-48b0-be74-de711496d948-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8506cc5b-22f3-48b0-be74-de711496d948" (UID: "8506cc5b-22f3-48b0-be74-de711496d948"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:50:04.497164 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.497122 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3355ae2f-9b9b-46ef-8212-651aa8884535-kube-api-access-56dlc" (OuterVolumeSpecName: "kube-api-access-56dlc") pod "3355ae2f-9b9b-46ef-8212-651aa8884535" (UID: "3355ae2f-9b9b-46ef-8212-651aa8884535"). InnerVolumeSpecName "kube-api-access-56dlc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:50:04.500083 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.500057 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8506cc5b-22f3-48b0-be74-de711496d948-kube-api-access-sm462" (OuterVolumeSpecName: "kube-api-access-sm462") pod "8506cc5b-22f3-48b0-be74-de711496d948" (UID: "8506cc5b-22f3-48b0-be74-de711496d948"). InnerVolumeSpecName "kube-api-access-sm462". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:50:04.596110 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.596073 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm462\" (UniqueName: \"kubernetes.io/projected/8506cc5b-22f3-48b0-be74-de711496d948-kube-api-access-sm462\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:04.596110 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.596106 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8506cc5b-22f3-48b0-be74-de711496d948-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:04.596110 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.596115 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56dlc\" (UniqueName: \"kubernetes.io/projected/3355ae2f-9b9b-46ef-8212-651aa8884535-kube-api-access-56dlc\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:04.626481 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.626386 2552 generic.go:356] "Generic (PLEG): container finished" podID="3355ae2f-9b9b-46ef-8212-651aa8884535" containerID="0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777" exitCode=0 Sep 15 07:50:04.626481 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.626440 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" Sep 15 07:50:04.626481 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.626461 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" event={"ID":"3355ae2f-9b9b-46ef-8212-651aa8884535","Type":"ContainerDied","Data":"0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777"} Sep 15 07:50:04.626777 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.626525 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh" event={"ID":"3355ae2f-9b9b-46ef-8212-651aa8884535","Type":"ContainerDied","Data":"d7cefe2d11ba888dcc9547be6bd41f1d679f7a7f04060b6e5f6243fead3212b4"} Sep 15 07:50:04.626777 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.626558 2552 scope.go:117] "RemoveContainer" containerID="0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777" Sep 15 07:50:04.628238 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.628208 2552 generic.go:356] "Generic (PLEG): container finished" podID="8506cc5b-22f3-48b0-be74-de711496d948" containerID="86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f" exitCode=0 Sep 15 07:50:04.628345 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.628280 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" Sep 15 07:50:04.628454 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.628432 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" event={"ID":"8506cc5b-22f3-48b0-be74-de711496d948","Type":"ContainerDied","Data":"86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f"} Sep 15 07:50:04.628623 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.628589 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk" event={"ID":"8506cc5b-22f3-48b0-be74-de711496d948","Type":"ContainerDied","Data":"18c20085e7baca73a7275c08818ab51317bb2bfeac9eaaee54b3ac583e27cd54"} Sep 15 07:50:04.639518 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.639495 2552 scope.go:117] "RemoveContainer" containerID="0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777" Sep 15 07:50:04.639988 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:50:04.639959 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777\": container with ID starting with 0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777 not found: ID does not exist" containerID="0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777" Sep 15 07:50:04.640072 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.640000 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777"} err="failed to get container status \"0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777\": rpc error: code = NotFound desc = could not find container \"0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777\": container with ID starting with 0c3e8d6aa71efafed42ebae530f8986af2932d595a418804c8675e7e4f81a777 not found: ID does not exist" Sep 15 07:50:04.640072 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.640025 2552 scope.go:117] "RemoveContainer" containerID="86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f" Sep 15 07:50:04.648646 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.648630 2552 scope.go:117] "RemoveContainer" containerID="86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f" Sep 15 07:50:04.648954 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:50:04.648923 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f\": container with ID starting with 86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f not found: ID does not exist" containerID="86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f" Sep 15 07:50:04.649030 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.648964 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f"} err="failed to get container status \"86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f\": rpc error: code = NotFound desc = could not find container \"86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f\": container with ID starting with 86a8664faaa1246c2d9ef43a3fb88889b05417e0c0ed40f6dd6888766e0a133f not found: ID does not exist" Sep 15 07:50:04.655461 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.655409 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh"] Sep 15 07:50:04.658023 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.658001 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3a4c9f-5645d74986-mprzh"] Sep 15 07:50:04.661306 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.661286 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:04.666092 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.666068 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk"] Sep 15 07:50:04.668477 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.668458 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3a4c9f-88fbf8fb5-bhnxk"] Sep 15 07:50:04.759857 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:04.759756 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:05.474433 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:05.474405 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3355ae2f-9b9b-46ef-8212-651aa8884535" path="/var/lib/kubelet/pods/3355ae2f-9b9b-46ef-8212-651aa8884535/volumes" Sep 15 07:50:05.474798 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:05.474761 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8506cc5b-22f3-48b0-be74-de711496d948" path="/var/lib/kubelet/pods/8506cc5b-22f3-48b0-be74-de711496d948/volumes" Sep 15 07:50:05.475081 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:05.475068 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b08213a1-594b-4f37-84f8-d0b923248cd4" path="/var/lib/kubelet/pods/b08213a1-594b-4f37-84f8-d0b923248cd4/volumes" Sep 15 07:50:05.571328 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:05.571290 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:06.452386 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:06.452352 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:07.189984 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:07.189946 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:08.236583 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:08.236548 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:11.321721 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.321629 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g"] Sep 15 07:50:11.322338 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.322082 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" containerID="cri-o://e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780" gracePeriod=30 Sep 15 07:50:11.391485 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.391447 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:11.668398 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.668310 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr"] Sep 15 07:50:11.668773 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.668755 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b08213a1-594b-4f37-84f8-d0b923248cd4" containerName="revoke-keys" Sep 15 07:50:11.668897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.668778 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="b08213a1-594b-4f37-84f8-d0b923248cd4" containerName="revoke-keys" Sep 15 07:50:11.668897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.668809 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8506cc5b-22f3-48b0-be74-de711496d948" containerName="payload-processing" Sep 15 07:50:11.668897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.668818 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="8506cc5b-22f3-48b0-be74-de711496d948" containerName="payload-processing" Sep 15 07:50:11.668897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.668842 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3355ae2f-9b9b-46ef-8212-651aa8884535" containerName="payload-pre-processing" Sep 15 07:50:11.668897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.668851 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3355ae2f-9b9b-46ef-8212-651aa8884535" containerName="payload-pre-processing" Sep 15 07:50:11.669166 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.668970 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="8506cc5b-22f3-48b0-be74-de711496d948" containerName="payload-processing" Sep 15 07:50:11.669166 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.668986 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3355ae2f-9b9b-46ef-8212-651aa8884535" containerName="payload-pre-processing" Sep 15 07:50:11.669166 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.669000 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="b08213a1-594b-4f37-84f8-d0b923248cd4" containerName="revoke-keys" Sep 15 07:50:11.673511 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.673491 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr" Sep 15 07:50:11.675798 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.675767 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 07:50:11.675958 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.675766 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8jpwk\"" Sep 15 07:50:11.678920 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.678892 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr"] Sep 15 07:50:11.708531 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.708479 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:11.768504 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.768476 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73e37a42-eb31-4056-90a5-c9eb15eb0bb8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr\" (UID: \"73e37a42-eb31-4056-90a5-c9eb15eb0bb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr" Sep 15 07:50:11.845219 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.845169 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:11.869608 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.869576 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73e37a42-eb31-4056-90a5-c9eb15eb0bb8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr\" (UID: \"73e37a42-eb31-4056-90a5-c9eb15eb0bb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr" Sep 15 07:50:11.870229 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.870212 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73e37a42-eb31-4056-90a5-c9eb15eb0bb8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr\" (UID: \"73e37a42-eb31-4056-90a5-c9eb15eb0bb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr" Sep 15 07:50:11.986049 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:11.986020 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr" Sep 15 07:50:12.255293 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:12.255266 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr"] Sep 15 07:50:12.256671 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:50:12.256641 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e37a42_eb31_4056_90a5_c9eb15eb0bb8.slice/crio-828e51dbcf94bb0795a85b3c16a8c7fd0acd63918d0b32858542e7a1c77c2c0b WatchSource:0}: Error finding container 828e51dbcf94bb0795a85b3c16a8c7fd0acd63918d0b32858542e7a1c77c2c0b: Status 404 returned error can't find the container with id 828e51dbcf94bb0795a85b3c16a8c7fd0acd63918d0b32858542e7a1c77c2c0b Sep 15 07:50:12.675447 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:12.675359 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr" event={"ID":"73e37a42-eb31-4056-90a5-c9eb15eb0bb8","Type":"ContainerStarted","Data":"e23e5bd2db7b5df7310317fa758271cf4d16e07c88b02ed8051363dc6c81f97c"} Sep 15 07:50:12.675447 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:12.675416 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr" event={"ID":"73e37a42-eb31-4056-90a5-c9eb15eb0bb8","Type":"ContainerStarted","Data":"828e51dbcf94bb0795a85b3c16a8c7fd0acd63918d0b32858542e7a1c77c2c0b"} Sep 15 07:50:12.690920 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:12.690878 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr" podStartSLOduration=1.69084819 podStartE2EDuration="1.69084819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:50:12.689325432 +0000 UTC m=+1893.670487601" watchObservedRunningTime="2026-09-15 07:50:12.69084819 +0000 UTC m=+1893.672010357" Sep 15 07:50:12.733408 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:12.733363 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:13.565566 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:13.565534 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:13.682373 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:13.682335 2552 generic.go:356] "Generic (PLEG): container finished" podID="73e37a42-eb31-4056-90a5-c9eb15eb0bb8" containerID="e23e5bd2db7b5df7310317fa758271cf4d16e07c88b02ed8051363dc6c81f97c" exitCode=0 Sep 15 07:50:13.682765 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:13.682371 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr" event={"ID":"73e37a42-eb31-4056-90a5-c9eb15eb0bb8","Type":"ContainerDied","Data":"e23e5bd2db7b5df7310317fa758271cf4d16e07c88b02ed8051363dc6c81f97c"} Sep 15 07:50:14.311548 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:14.311519 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:14.819799 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:14.819774 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr" Sep 15 07:50:15.001532 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:15.001501 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73e37a42-eb31-4056-90a5-c9eb15eb0bb8-maas-api-service-ca\") pod \"73e37a42-eb31-4056-90a5-c9eb15eb0bb8\" (UID: \"73e37a42-eb31-4056-90a5-c9eb15eb0bb8\") " Sep 15 07:50:15.001907 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:15.001853 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73e37a42-eb31-4056-90a5-c9eb15eb0bb8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73e37a42-eb31-4056-90a5-c9eb15eb0bb8" (UID: "73e37a42-eb31-4056-90a5-c9eb15eb0bb8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:50:15.103175 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:15.103137 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73e37a42-eb31-4056-90a5-c9eb15eb0bb8-maas-api-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:15.182913 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:15.182877 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:15.695195 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:15.695167 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr" Sep 15 07:50:15.695195 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:15.695180 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr" event={"ID":"73e37a42-eb31-4056-90a5-c9eb15eb0bb8","Type":"ContainerDied","Data":"828e51dbcf94bb0795a85b3c16a8c7fd0acd63918d0b32858542e7a1c77c2c0b"} Sep 15 07:50:15.695483 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:15.695244 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="828e51dbcf94bb0795a85b3c16a8c7fd0acd63918d0b32858542e7a1c77c2c0b" Sep 15 07:50:15.920380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:15.920289 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:16.579537 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.579512 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:50:16.702290 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.702202 2552 generic.go:356] "Generic (PLEG): container finished" podID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerID="e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780" exitCode=0 Sep 15 07:50:16.702290 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.702233 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" event={"ID":"d69777a3-ea17-44f1-b145-f2f9de1f1318","Type":"ContainerDied","Data":"e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780"} Sep 15 07:50:16.702290 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.702257 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" Sep 15 07:50:16.702548 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.702302 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g" event={"ID":"d69777a3-ea17-44f1-b145-f2f9de1f1318","Type":"ContainerDied","Data":"846fa308eac3b67d663b4de26a8b066bd7b47f4cd06c9d606baf74db7bed0557"} Sep 15 07:50:16.702548 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.702332 2552 scope.go:117] "RemoveContainer" containerID="e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780" Sep 15 07:50:16.711515 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.711498 2552 scope.go:117] "RemoveContainer" containerID="723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f" Sep 15 07:50:16.719241 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719225 2552 scope.go:117] "RemoveContainer" containerID="e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780" Sep 15 07:50:16.719474 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719459 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-podinfo\") pod \"d69777a3-ea17-44f1-b145-f2f9de1f1318\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " Sep 15 07:50:16.719518 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719491 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-envoy\") pod \"d69777a3-ea17-44f1-b145-f2f9de1f1318\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " Sep 15 07:50:16.719518 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719511 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-workload-certs\") pod \"d69777a3-ea17-44f1-b145-f2f9de1f1318\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " Sep 15 07:50:16.719596 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:50:16.719511 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780\": container with ID starting with e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780 not found: ID does not exist" containerID="e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780" Sep 15 07:50:16.719596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719536 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780"} err="failed to get container status \"e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780\": rpc error: code = NotFound desc = could not find container \"e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780\": container with ID starting with e4d786410c82e2c6057d4472cd01987b1e00e6402f6e67190ba78d87a2b20780 not found: ID does not exist" Sep 15 07:50:16.719596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719548 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-credential-socket\") pod \"d69777a3-ea17-44f1-b145-f2f9de1f1318\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " Sep 15 07:50:16.719596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719555 2552 scope.go:117] "RemoveContainer" containerID="723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f" Sep 15 07:50:16.719596 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719574 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqbjx\" (UniqueName: \"kubernetes.io/projected/d69777a3-ea17-44f1-b145-f2f9de1f1318-kube-api-access-lqbjx\") pod \"d69777a3-ea17-44f1-b145-f2f9de1f1318\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " Sep 15 07:50:16.719827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719644 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-token\") pod \"d69777a3-ea17-44f1-b145-f2f9de1f1318\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " Sep 15 07:50:16.719827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719684 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-data\") pod \"d69777a3-ea17-44f1-b145-f2f9de1f1318\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " Sep 15 07:50:16.719827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719727 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-workload-socket\") pod \"d69777a3-ea17-44f1-b145-f2f9de1f1318\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " Sep 15 07:50:16.719827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719775 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d69777a3-ea17-44f1-b145-f2f9de1f1318" (UID: "d69777a3-ea17-44f1-b145-f2f9de1f1318"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:50:16.719827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719795 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d69777a3-ea17-44f1-b145-f2f9de1f1318-istiod-ca-cert\") pod \"d69777a3-ea17-44f1-b145-f2f9de1f1318\" (UID: \"d69777a3-ea17-44f1-b145-f2f9de1f1318\") " Sep 15 07:50:16.720103 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719844 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d69777a3-ea17-44f1-b145-f2f9de1f1318" (UID: "d69777a3-ea17-44f1-b145-f2f9de1f1318"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:50:16.720103 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.719975 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-data" (OuterVolumeSpecName: "istio-data") pod "d69777a3-ea17-44f1-b145-f2f9de1f1318" (UID: "d69777a3-ea17-44f1-b145-f2f9de1f1318"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:50:16.720366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.720251 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d69777a3-ea17-44f1-b145-f2f9de1f1318-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d69777a3-ea17-44f1-b145-f2f9de1f1318" (UID: "d69777a3-ea17-44f1-b145-f2f9de1f1318"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:50:16.720366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.720272 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d69777a3-ea17-44f1-b145-f2f9de1f1318" (UID: "d69777a3-ea17-44f1-b145-f2f9de1f1318"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:50:16.720366 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:50:16.720282 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f\": container with ID starting with 723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f not found: ID does not exist" containerID="723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f" Sep 15 07:50:16.720366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.720336 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f"} err="failed to get container status \"723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f\": rpc error: code = NotFound desc = could not find container \"723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f\": container with ID starting with 723a45dbef056339d4ed6d881609273d0941458a18cdabdb23dded226348047f not found: ID does not exist" Sep 15 07:50:16.720613 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.720574 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-data\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:16.720613 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.720595 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-workload-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:16.720613 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.720610 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d69777a3-ea17-44f1-b145-f2f9de1f1318-istiod-ca-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:16.720782 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.720624 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-workload-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:16.720782 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.720640 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-credential-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:16.721690 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.721660 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d69777a3-ea17-44f1-b145-f2f9de1f1318" (UID: "d69777a3-ea17-44f1-b145-f2f9de1f1318"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 07:50:16.721792 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.721765 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d69777a3-ea17-44f1-b145-f2f9de1f1318" (UID: "d69777a3-ea17-44f1-b145-f2f9de1f1318"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:50:16.721849 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.721834 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-token" (OuterVolumeSpecName: "istio-token") pod "d69777a3-ea17-44f1-b145-f2f9de1f1318" (UID: "d69777a3-ea17-44f1-b145-f2f9de1f1318"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:50:16.721991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.721975 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d69777a3-ea17-44f1-b145-f2f9de1f1318-kube-api-access-lqbjx" (OuterVolumeSpecName: "kube-api-access-lqbjx") pod "d69777a3-ea17-44f1-b145-f2f9de1f1318" (UID: "d69777a3-ea17-44f1-b145-f2f9de1f1318"). InnerVolumeSpecName "kube-api-access-lqbjx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:50:16.821223 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.821187 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-podinfo\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:16.821223 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.821217 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-envoy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:16.821223 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.821226 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqbjx\" (UniqueName: \"kubernetes.io/projected/d69777a3-ea17-44f1-b145-f2f9de1f1318-kube-api-access-lqbjx\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:16.821223 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:16.821236 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d69777a3-ea17-44f1-b145-f2f9de1f1318-istio-token\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:17.024432 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:17.024403 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g"] Sep 15 07:50:17.029199 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:17.029172 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3a4c9f-openshift-default-56d5bf5d-xhp8g"] Sep 15 07:50:17.474581 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:17.474558 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" path="/var/lib/kubelet/pods/d69777a3-ea17-44f1-b145-f2f9de1f1318/volumes" Sep 15 07:50:22.198716 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:22.198677 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:22.199125 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:22.198802 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-27b9d8-29824305-xqhcg"] Sep 15 07:50:22.202787 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:22.202763 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-27b9d8-29824305-xqhcg"] Sep 15 07:50:22.268580 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:22.268548 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:22.463468 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:22.463385 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:22.484987 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:22.484955 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr"] Sep 15 07:50:22.488392 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:22.488366 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-27b9d8-z46wr"] Sep 15 07:50:22.530300 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:22.530269 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:22.808145 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:22.808091 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:22.915726 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:22.915689 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:23.474726 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:23.474683 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e2be7f1-b12f-4f1d-9b97-086f223850ee" path="/var/lib/kubelet/pods/3e2be7f1-b12f-4f1d-9b97-086f223850ee/volumes" Sep 15 07:50:23.475224 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:23.475144 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73e37a42-eb31-4056-90a5-c9eb15eb0bb8" path="/var/lib/kubelet/pods/73e37a42-eb31-4056-90a5-c9eb15eb0bb8/volumes" Sep 15 07:50:23.643738 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:23.643703 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:24.376555 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:24.376524 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:25.155792 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:25.155760 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:28.487389 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:28.487355 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:31.482779 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:31.482741 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:31.538915 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:31.538882 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:45.331721 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.331679 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:45.428543 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.428512 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks"] Sep 15 07:50:45.428891 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.428879 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" Sep 15 07:50:45.428939 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.428893 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" Sep 15 07:50:45.428939 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.428905 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73e37a42-eb31-4056-90a5-c9eb15eb0bb8" containerName="revoke-keys" Sep 15 07:50:45.428939 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.428910 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="73e37a42-eb31-4056-90a5-c9eb15eb0bb8" containerName="revoke-keys" Sep 15 07:50:45.428939 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.428917 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" Sep 15 07:50:45.428939 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.428924 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" Sep 15 07:50:45.429098 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.429010 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" Sep 15 07:50:45.429098 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.429019 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="73e37a42-eb31-4056-90a5-c9eb15eb0bb8" containerName="revoke-keys" Sep 15 07:50:45.431978 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.431951 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks" Sep 15 07:50:45.434190 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.434169 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 07:50:45.434318 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.434171 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8jpwk\"" Sep 15 07:50:45.438497 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.438474 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks"] Sep 15 07:50:45.482360 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.482331 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:45.588233 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.588152 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45a4ba8d-2c95-436e-874c-ef49e9e5c630-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks\" (UID: \"45a4ba8d-2c95-436e-874c-ef49e9e5c630\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks" Sep 15 07:50:45.689343 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.689309 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45a4ba8d-2c95-436e-874c-ef49e9e5c630-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks\" (UID: \"45a4ba8d-2c95-436e-874c-ef49e9e5c630\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks" Sep 15 07:50:45.689902 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.689859 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45a4ba8d-2c95-436e-874c-ef49e9e5c630-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks\" (UID: \"45a4ba8d-2c95-436e-874c-ef49e9e5c630\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks" Sep 15 07:50:45.744312 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.744284 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks" Sep 15 07:50:45.755270 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:45.755236 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:46.025996 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:46.025965 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks"] Sep 15 07:50:46.028062 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:50:46.028035 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45a4ba8d_2c95_436e_874c_ef49e9e5c630.slice/crio-74d7f01ea5f72a687755bae7dd52adbc2a467ac1449f68307e22b0c82381e419 WatchSource:0}: Error finding container 74d7f01ea5f72a687755bae7dd52adbc2a467ac1449f68307e22b0c82381e419: Status 404 returned error can't find the container with id 74d7f01ea5f72a687755bae7dd52adbc2a467ac1449f68307e22b0c82381e419 Sep 15 07:50:46.651603 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:46.651509 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:46.862436 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:46.862400 2552 generic.go:356] "Generic (PLEG): container finished" podID="45a4ba8d-2c95-436e-874c-ef49e9e5c630" containerID="9c587c651e424af92f0b78c6f33e38be2ef69717daf7ab87c5aab77283d359b5" exitCode=0 Sep 15 07:50:46.862589 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:46.862458 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks" event={"ID":"45a4ba8d-2c95-436e-874c-ef49e9e5c630","Type":"ContainerDied","Data":"9c587c651e424af92f0b78c6f33e38be2ef69717daf7ab87c5aab77283d359b5"} Sep 15 07:50:46.862589 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:46.862500 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks" event={"ID":"45a4ba8d-2c95-436e-874c-ef49e9e5c630","Type":"ContainerStarted","Data":"74d7f01ea5f72a687755bae7dd52adbc2a467ac1449f68307e22b0c82381e419"} Sep 15 07:50:47.381795 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:47.381762 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:48.000352 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:48.000327 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks" Sep 15 07:50:48.112178 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:48.112148 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45a4ba8d-2c95-436e-874c-ef49e9e5c630-maas-api-service-ca\") pod \"45a4ba8d-2c95-436e-874c-ef49e9e5c630\" (UID: \"45a4ba8d-2c95-436e-874c-ef49e9e5c630\") " Sep 15 07:50:48.112490 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:48.112466 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a4ba8d-2c95-436e-874c-ef49e9e5c630-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "45a4ba8d-2c95-436e-874c-ef49e9e5c630" (UID: "45a4ba8d-2c95-436e-874c-ef49e9e5c630"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:50:48.204295 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:48.204200 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:48.213098 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:48.213073 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45a4ba8d-2c95-436e-874c-ef49e9e5c630-maas-api-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:50:48.878658 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:48.878621 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks" Sep 15 07:50:48.878849 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:48.878661 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks" event={"ID":"45a4ba8d-2c95-436e-874c-ef49e9e5c630","Type":"ContainerDied","Data":"74d7f01ea5f72a687755bae7dd52adbc2a467ac1449f68307e22b0c82381e419"} Sep 15 07:50:48.878849 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:48.878721 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74d7f01ea5f72a687755bae7dd52adbc2a467ac1449f68307e22b0c82381e419" Sep 15 07:50:48.936563 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:48.936536 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:50:50.304276 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:50.304241 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck"] Sep 15 07:50:50.379363 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:50:50.379321 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:01.141303 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:01.141268 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr"] Sep 15 07:51:01.146401 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:01.146372 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-f56165-29824305-n76kr"] Sep 15 07:51:01.146554 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:01.146392 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:01.244368 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:01.244328 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:01.323311 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:01.323272 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks"] Sep 15 07:51:01.325429 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:01.325395 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:01.326066 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:01.326049 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f56165-4zkks"] Sep 15 07:51:01.390292 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:01.390264 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:01.475757 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:01.475732 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e5b8698-3bac-43f6-8428-be56b15d8405" path="/var/lib/kubelet/pods/1e5b8698-3bac-43f6-8428-be56b15d8405/volumes" Sep 15 07:51:01.476067 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:01.476054 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45a4ba8d-2c95-436e-874c-ef49e9e5c630" path="/var/lib/kubelet/pods/45a4ba8d-2c95-436e-874c-ef49e9e5c630/volumes" Sep 15 07:51:01.806136 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:01.806049 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:01.956780 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:01.956743 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:02.532997 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:02.532964 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:03.351280 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:03.351236 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:05.324018 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.323985 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" podUID="f3852c36-8497-4de4-bbef-717d8d7cd96d" containerName="main" containerID="cri-o://c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094" gracePeriod=45 Sep 15 07:51:05.353385 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:51:05.353361 2552 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3852c36_8497_4de4_bbef_717d8d7cd96d.slice/crio-c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094.scope\": RecentStats: unable to find data in memory cache]" Sep 15 07:51:05.574917 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.574827 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:51:05.680097 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.680068 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pmnk\" (UniqueName: \"kubernetes.io/projected/f3852c36-8497-4de4-bbef-717d8d7cd96d-kube-api-access-2pmnk\") pod \"f3852c36-8497-4de4-bbef-717d8d7cd96d\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " Sep 15 07:51:05.680300 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.680128 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-dshm\") pod \"f3852c36-8497-4de4-bbef-717d8d7cd96d\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " Sep 15 07:51:05.680300 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.680205 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-tmp-dir\") pod \"f3852c36-8497-4de4-bbef-717d8d7cd96d\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " Sep 15 07:51:05.680300 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.680235 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3852c36-8497-4de4-bbef-717d8d7cd96d-tls-certs\") pod \"f3852c36-8497-4de4-bbef-717d8d7cd96d\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " Sep 15 07:51:05.680300 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.680286 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-model-cache\") pod \"f3852c36-8497-4de4-bbef-717d8d7cd96d\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " Sep 15 07:51:05.680514 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.680339 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-home\") pod \"f3852c36-8497-4de4-bbef-717d8d7cd96d\" (UID: \"f3852c36-8497-4de4-bbef-717d8d7cd96d\") " Sep 15 07:51:05.680569 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.680527 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f3852c36-8497-4de4-bbef-717d8d7cd96d" (UID: "f3852c36-8497-4de4-bbef-717d8d7cd96d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:51:05.680617 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.680563 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-model-cache" (OuterVolumeSpecName: "model-cache") pod "f3852c36-8497-4de4-bbef-717d8d7cd96d" (UID: "f3852c36-8497-4de4-bbef-717d8d7cd96d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:51:05.680723 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.680705 2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-model-cache\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:51:05.680785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.680718 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-home" (OuterVolumeSpecName: "home") pod "f3852c36-8497-4de4-bbef-717d8d7cd96d" (UID: "f3852c36-8497-4de4-bbef-717d8d7cd96d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:51:05.680785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.680729 2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-tmp-dir\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:51:05.682844 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.682812 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3852c36-8497-4de4-bbef-717d8d7cd96d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3852c36-8497-4de4-bbef-717d8d7cd96d" (UID: "f3852c36-8497-4de4-bbef-717d8d7cd96d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:51:05.682844 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.682818 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3852c36-8497-4de4-bbef-717d8d7cd96d-kube-api-access-2pmnk" (OuterVolumeSpecName: "kube-api-access-2pmnk") pod "f3852c36-8497-4de4-bbef-717d8d7cd96d" (UID: "f3852c36-8497-4de4-bbef-717d8d7cd96d"). InnerVolumeSpecName "kube-api-access-2pmnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:51:05.683012 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.682857 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-dshm" (OuterVolumeSpecName: "dshm") pod "f3852c36-8497-4de4-bbef-717d8d7cd96d" (UID: "f3852c36-8497-4de4-bbef-717d8d7cd96d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:51:05.781413 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.781370 2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3852c36-8497-4de4-bbef-717d8d7cd96d-tls-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:51:05.781413 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.781404 2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-home\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:51:05.781413 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.781415 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pmnk\" (UniqueName: \"kubernetes.io/projected/f3852c36-8497-4de4-bbef-717d8d7cd96d-kube-api-access-2pmnk\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:51:05.781413 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.781424 2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3852c36-8497-4de4-bbef-717d8d7cd96d-dshm\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:51:05.966980 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.966893 2552 generic.go:356] "Generic (PLEG): container finished" podID="f3852c36-8497-4de4-bbef-717d8d7cd96d" containerID="c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094" exitCode=0 Sep 15 07:51:05.966980 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.966928 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" Sep 15 07:51:05.966980 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.966974 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" event={"ID":"f3852c36-8497-4de4-bbef-717d8d7cd96d","Type":"ContainerDied","Data":"c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094"} Sep 15 07:51:05.967194 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.967013 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck" event={"ID":"f3852c36-8497-4de4-bbef-717d8d7cd96d","Type":"ContainerDied","Data":"a5ebdf4622e3742759fe542636581bf60d392fc1a040ddee682c287d9308481a"} Sep 15 07:51:05.967194 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.967031 2552 scope.go:117] "RemoveContainer" containerID="c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094" Sep 15 07:51:05.976292 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.976272 2552 scope.go:117] "RemoveContainer" containerID="c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094" Sep 15 07:51:05.976558 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:51:05.976536 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094\": container with ID starting with c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094 not found: ID does not exist" containerID="c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094" Sep 15 07:51:05.976658 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.976565 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094"} err="failed to get container status \"c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094\": rpc error: code = NotFound desc = could not find container \"c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094\": container with ID starting with c44dd8d29351c3cc5096ac42667e443af4aefaf824a921056d790587d5d05094 not found: ID does not exist" Sep 15 07:51:05.987154 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.987129 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck"] Sep 15 07:51:05.993373 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:05.993353 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-f56165/distinct-2-w4-f56165-kserve-5b9dd5ffd5-9mlck"] Sep 15 07:51:07.474534 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:07.474500 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3852c36-8497-4de4-bbef-717d8d7cd96d" path="/var/lib/kubelet/pods/f3852c36-8497-4de4-bbef-717d8d7cd96d/volumes" Sep 15 07:51:07.526345 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:07.526314 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:07.624392 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:07.624359 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:07.741443 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:07.741357 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:11.818451 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:11.818413 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:11.919346 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:11.919309 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:20.814326 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:20.814287 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:22.418419 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:22.418382 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:26.924446 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:26.924410 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:27.078085 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:27.078041 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:27.175479 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:27.175400 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:28.143440 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:28.143395 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:33.793375 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:33.793295 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:51:39.725438 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:39.725401 2552 scope.go:117] "RemoveContainer" containerID="8b75957b9f5917f8d712c08646bfc396dde8c19e70418b3e6dcc4df78c08435d" Sep 15 07:51:39.733656 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:39.733635 2552 scope.go:117] "RemoveContainer" containerID="7f41951b4556a00d4922620494bb22f1d54213d885b04af8b3166a46b1f17ab9" Sep 15 07:51:39.741216 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:39.741200 2552 scope.go:117] "RemoveContainer" containerID="dba5c75fdec09d88f9a01313f2b7548a5ccee3e17b675ff6802bc50665f92907" Sep 15 07:51:43.531792 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:51:43.531761 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:52:30.920859 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:52:30.920823 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:52:32.012417 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:52:32.012379 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:52:32.062418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:52:32.062379 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:52:43.239557 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:52:43.239511 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:53:20.935168 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:53:20.935136 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:53:28.489602 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:53:28.489563 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:53:28.556390 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:53:28.556361 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:53:41.148491 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:53:41.148463 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:53:41.155236 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:53:41.155210 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:53:41.155734 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:53:41.155717 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:53:41.162925 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:53:41.162907 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:53:47.736882 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:53:47.736835 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:01.934458 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:01.934421 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:02.940540 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:02.940506 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:03.352739 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:03.352701 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:03.640372 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:03.640273 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:03.788841 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:03.788811 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:06.618987 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:06.618962 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:06.883076 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:06.883001 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:07.004096 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:07.004064 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:07.111035 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:07.111003 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:07.646806 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:07.646770 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:08.033623 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:08.033585 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:08.368469 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:08.368374 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:08.472515 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:08.472477 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:08.931293 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:08.931260 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:09.032574 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:09.032536 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:09.114518 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:09.114485 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:21.165042 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:21.165009 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:30.976227 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:30.976188 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:31.077100 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:31.077057 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:31.572189 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:31.572156 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:31.673356 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:31.673318 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:32.296381 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:32.296344 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:32.440983 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:32.440954 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:32.606665 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:32.606580 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:32.837212 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:32.837179 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:33.250752 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:33.250708 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:35.351424 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:35.351376 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:36.062568 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:36.062533 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:36.159598 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:36.159564 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:36.300610 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:36.300573 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:37.037063 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:37.037027 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:37.830374 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:37.830338 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:38.847756 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:38.847722 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:38.949394 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:38.949360 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:39.439584 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:39.439550 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:39.542102 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:39.542068 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:39.731643 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:39.731613 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:54:39.786027 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:54:39.786000 2552 scope.go:117] "RemoveContainer" containerID="a815cfeb29412fd3fe81c77e5a40fdf9326686e2f028b37fd3a3774c33d25909" Sep 15 07:55:03.139166 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:03.139127 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:03.239660 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:03.239627 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:03.841659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:03.841618 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:04.561384 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:04.561345 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:05.349980 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:05.349951 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:06.036335 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:06.036298 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:10.998928 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:10.998894 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-cgt48_11660574-efac-4686-962b-934e3bd76287/payload-processing/0.log" Sep 15 07:55:19.839346 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:19.839308 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:39.806662 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:39.806624 2552 scope.go:117] "RemoveContainer" containerID="615c8137c36497896abc64b076c71197b377e31fd5d7ba8e96964d919c0c574c" Sep 15 07:55:44.238899 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.238848 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-cgt48_11660574-efac-4686-962b-934e3bd76287/payload-processing/0.log" Sep 15 07:55:44.613745 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.613651 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:44.899666 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.899589 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc"] Sep 15 07:55:44.900214 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.900198 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a4ba8d-2c95-436e-874c-ef49e9e5c630" containerName="revoke-keys" Sep 15 07:55:44.900271 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.900218 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a4ba8d-2c95-436e-874c-ef49e9e5c630" containerName="revoke-keys" Sep 15 07:55:44.900271 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.900239 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3852c36-8497-4de4-bbef-717d8d7cd96d" containerName="main" Sep 15 07:55:44.900271 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.900248 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3852c36-8497-4de4-bbef-717d8d7cd96d" containerName="main" Sep 15 07:55:44.900365 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.900342 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3852c36-8497-4de4-bbef-717d8d7cd96d" containerName="main" Sep 15 07:55:44.900365 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.900356 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d69777a3-ea17-44f1-b145-f2f9de1f1318" containerName="istio-proxy" Sep 15 07:55:44.900435 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.900367 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a4ba8d-2c95-436e-874c-ef49e9e5c630" containerName="revoke-keys" Sep 15 07:55:44.903743 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.903726 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:44.907679 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.907661 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-0fd0de-openshift-default-dockercfg-rrxs4\"" Sep 15 07:55:44.917043 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.917020 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc"] Sep 15 07:55:44.930492 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.930453 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:44.966144 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.966114 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-credential-socket\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:44.966301 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.966159 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-workload-certs\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:44.966301 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.966181 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istiod-ca-cert\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:44.966301 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.966253 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-workload-socket\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:44.966301 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.966274 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-envoy\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:44.966301 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.966299 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-podinfo\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:44.966480 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.966322 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qgf5\" (UniqueName: \"kubernetes.io/projected/93ec2b06-ade3-4ea1-9656-e85fa32022c9-kube-api-access-4qgf5\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:44.966480 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.966361 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-data\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:44.966480 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:44.966382 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-token\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.067823 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.067785 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-credential-socket\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068019 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.067843 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-workload-certs\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068019 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.067905 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istiod-ca-cert\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068019 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.067937 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-workload-socket\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068019 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.067968 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-envoy\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068019 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.067992 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-podinfo\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068302 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.068021 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qgf5\" (UniqueName: \"kubernetes.io/projected/93ec2b06-ade3-4ea1-9656-e85fa32022c9-kube-api-access-4qgf5\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068302 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.068080 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-data\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068302 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.068108 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-token\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068302 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.068234 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-credential-socket\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068511 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.068437 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-workload-certs\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068569 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.068537 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-workload-socket\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068657 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.068637 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-data\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.068856 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.068835 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istiod-ca-cert\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.070417 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.070393 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-envoy\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.070541 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.070522 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-podinfo\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.078321 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.078299 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-token\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.080234 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.080214 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qgf5\" (UniqueName: \"kubernetes.io/projected/93ec2b06-ade3-4ea1-9656-e85fa32022c9-kube-api-access-4qgf5\") pod \"e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.189313 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.189237 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:45.213931 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.213902 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:45.496718 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.496616 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc"] Sep 15 07:55:45.499070 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:55:45.499041 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93ec2b06_ade3_4ea1_9656_e85fa32022c9.slice/crio-168a609b8c93891665653d6612fcef4286f0815b679c5743252d6c8b85bbebfc WatchSource:0}: Error finding container 168a609b8c93891665653d6612fcef4286f0815b679c5743252d6c8b85bbebfc: Status 404 returned error can't find the container with id 168a609b8c93891665653d6612fcef4286f0815b679c5743252d6c8b85bbebfc Sep 15 07:55:45.501315 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.501278 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:55:45.501415 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.501341 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:55:45.501415 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:45.501369 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:55:46.412660 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:46.412624 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" event={"ID":"93ec2b06-ade3-4ea1-9656-e85fa32022c9","Type":"ContainerStarted","Data":"08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa"} Sep 15 07:55:46.412841 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:46.412671 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" event={"ID":"93ec2b06-ade3-4ea1-9656-e85fa32022c9","Type":"ContainerStarted","Data":"168a609b8c93891665653d6612fcef4286f0815b679c5743252d6c8b85bbebfc"} Sep 15 07:55:46.433584 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:46.433527 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" podStartSLOduration=2.433511861 podStartE2EDuration="2.433511861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:55:46.431241489 +0000 UTC m=+2227.412403656" watchObservedRunningTime="2026-09-15 07:55:46.433511861 +0000 UTC m=+2227.414674027" Sep 15 07:55:47.214423 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:47.214384 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:47.220145 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:47.220118 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:47.417280 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:47.417249 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:47.418165 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:47.418147 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:55:47.637189 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:47.637150 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:48.322652 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:48.322618 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:48.420413 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:48.420379 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:48.521706 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:48.521674 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:49.707661 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:49.707622 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:50.421991 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:50.421959 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:51.235474 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:51.235418 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:52.049608 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.049578 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:52.149826 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.149786 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:52.250952 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.250914 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:52.524767 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.524734 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:52.625081 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.625049 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:52.665771 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.665739 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49"] Sep 15 07:55:52.669451 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.669434 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" Sep 15 07:55:52.671567 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.671539 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-0fd0de-dockercfg-7plw7\"" Sep 15 07:55:52.671724 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.671699 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-0fd0de\"" Sep 15 07:55:52.680491 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.680465 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49"] Sep 15 07:55:52.741832 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.741790 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jdbn\" (UniqueName: \"kubernetes.io/projected/9905d4d3-b112-4d88-9a38-8728d84ab131-kube-api-access-8jdbn\") pod \"payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49\" (UID: \"9905d4d3-b112-4d88-9a38-8728d84ab131\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" Sep 15 07:55:52.742044 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.741967 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9905d4d3-b112-4d88-9a38-8728d84ab131-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49\" (UID: \"9905d4d3-b112-4d88-9a38-8728d84ab131\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" Sep 15 07:55:52.842601 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.842509 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9905d4d3-b112-4d88-9a38-8728d84ab131-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49\" (UID: \"9905d4d3-b112-4d88-9a38-8728d84ab131\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" Sep 15 07:55:52.842754 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.842605 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jdbn\" (UniqueName: \"kubernetes.io/projected/9905d4d3-b112-4d88-9a38-8728d84ab131-kube-api-access-8jdbn\") pod \"payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49\" (UID: \"9905d4d3-b112-4d88-9a38-8728d84ab131\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" Sep 15 07:55:52.843209 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.843190 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9905d4d3-b112-4d88-9a38-8728d84ab131-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49\" (UID: \"9905d4d3-b112-4d88-9a38-8728d84ab131\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" Sep 15 07:55:52.850041 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.850019 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jdbn\" (UniqueName: \"kubernetes.io/projected/9905d4d3-b112-4d88-9a38-8728d84ab131-kube-api-access-8jdbn\") pod \"payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49\" (UID: \"9905d4d3-b112-4d88-9a38-8728d84ab131\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" Sep 15 07:55:52.981936 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:52.981898 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" Sep 15 07:55:53.252163 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:53.252137 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49"] Sep 15 07:55:53.254479 ip-10-0-143-175 kubenswrapper[2552]: W0915 07:55:53.254452 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9905d4d3_b112_4d88_9a38_8728d84ab131.slice/crio-2cc5293ded41629f63a2b78a9b7391a0e8b2b3a4d8aa5d65d0f2b7294d7054f7 WatchSource:0}: Error finding container 2cc5293ded41629f63a2b78a9b7391a0e8b2b3a4d8aa5d65d0f2b7294d7054f7: Status 404 returned error can't find the container with id 2cc5293ded41629f63a2b78a9b7391a0e8b2b3a4d8aa5d65d0f2b7294d7054f7 Sep 15 07:55:53.450832 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:53.450799 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" event={"ID":"9905d4d3-b112-4d88-9a38-8728d84ab131","Type":"ContainerStarted","Data":"1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df"} Sep 15 07:55:53.451102 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:53.450854 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" event={"ID":"9905d4d3-b112-4d88-9a38-8728d84ab131","Type":"ContainerStarted","Data":"2cc5293ded41629f63a2b78a9b7391a0e8b2b3a4d8aa5d65d0f2b7294d7054f7"} Sep 15 07:55:53.451102 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:53.450907 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" Sep 15 07:55:53.454119 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:53.454093 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:55:53.474788 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:55:53.474737 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" podStartSLOduration=1.474713948 podStartE2EDuration="1.474713948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:55:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:55:53.473463891 +0000 UTC m=+2234.454626058" watchObservedRunningTime="2026-09-15 07:55:53.474713948 +0000 UTC m=+2234.455876119" Sep 15 07:56:03.553194 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:03.553159 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:04.457791 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:04.457759 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" Sep 15 07:56:07.532750 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:07.532720 2552 generic.go:356] "Generic (PLEG): container finished" podID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerID="08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa" exitCode=0 Sep 15 07:56:07.533294 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:07.532789 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" event={"ID":"93ec2b06-ade3-4ea1-9656-e85fa32022c9","Type":"ContainerDied","Data":"08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa"} Sep 15 07:56:07.533366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:07.533302 2552 scope.go:117] "RemoveContainer" containerID="08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa" Sep 15 07:56:07.534398 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:07.534371 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:56:07.534482 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:07.534437 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:56:07.534482 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:07.534466 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 15 07:56:08.539884 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:08.539846 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" event={"ID":"93ec2b06-ade3-4ea1-9656-e85fa32022c9","Type":"ContainerStarted","Data":"e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d"} Sep 15 07:56:09.215118 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:09.215082 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:56:09.215329 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:09.215298 2552 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body= Sep 15 07:56:09.215379 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:09.215357 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" Sep 15 07:56:10.214677 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:10.214633 2552 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body= Sep 15 07:56:10.215187 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:10.214698 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" Sep 15 07:56:11.214409 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:11.214374 2552 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body= Sep 15 07:56:11.214579 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:11.214430 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" Sep 15 07:56:12.214901 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:12.214844 2552 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body= Sep 15 07:56:12.215301 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:12.214915 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" Sep 15 07:56:13.215077 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:13.215041 2552 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body= Sep 15 07:56:13.215480 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:13.215099 2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" Sep 15 07:56:14.218653 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:14.218624 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:56:14.219075 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:14.218899 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:56:14.219659 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:14.219636 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:56:14.242354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:14.242326 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:14.959588 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:14.959556 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:15.658365 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:15.658334 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:16.419402 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:16.419370 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:17.150421 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:17.150387 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:17.857456 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:17.857423 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:18.439271 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:18.439237 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:20.058269 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:20.058237 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:20.808279 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:20.808243 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:21.625501 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:21.625470 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:22.429143 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:22.429111 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:23.138708 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:23.138675 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:23.953678 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:23.953637 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:30.131994 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.131948 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49"] Sep 15 07:56:30.132465 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.132214 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" podUID="9905d4d3-b112-4d88-9a38-8728d84ab131" containerName="payload-processing" containerID="cri-o://1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df" gracePeriod=30 Sep 15 07:56:30.133240 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.133211 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:30.231920 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.231887 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:30.322011 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.321911 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:30.386848 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.386792 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" Sep 15 07:56:30.410375 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.410351 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9905d4d3-b112-4d88-9a38-8728d84ab131-plugins-config-volume\") pod \"9905d4d3-b112-4d88-9a38-8728d84ab131\" (UID: \"9905d4d3-b112-4d88-9a38-8728d84ab131\") " Sep 15 07:56:30.410518 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.410501 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jdbn\" (UniqueName: \"kubernetes.io/projected/9905d4d3-b112-4d88-9a38-8728d84ab131-kube-api-access-8jdbn\") pod \"9905d4d3-b112-4d88-9a38-8728d84ab131\" (UID: \"9905d4d3-b112-4d88-9a38-8728d84ab131\") " Sep 15 07:56:30.410860 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.410714 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9905d4d3-b112-4d88-9a38-8728d84ab131-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9905d4d3-b112-4d88-9a38-8728d84ab131" (UID: "9905d4d3-b112-4d88-9a38-8728d84ab131"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:56:30.410986 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.410862 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9905d4d3-b112-4d88-9a38-8728d84ab131-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:56:30.411677 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.411643 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:30.413126 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.413100 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9905d4d3-b112-4d88-9a38-8728d84ab131-kube-api-access-8jdbn" (OuterVolumeSpecName: "kube-api-access-8jdbn") pod "9905d4d3-b112-4d88-9a38-8728d84ab131" (UID: "9905d4d3-b112-4d88-9a38-8728d84ab131"). InnerVolumeSpecName "kube-api-access-8jdbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:56:30.512366 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.512331 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jdbn\" (UniqueName: \"kubernetes.io/projected/9905d4d3-b112-4d88-9a38-8728d84ab131-kube-api-access-8jdbn\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:56:30.653789 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.653681 2552 generic.go:356] "Generic (PLEG): container finished" podID="9905d4d3-b112-4d88-9a38-8728d84ab131" containerID="1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df" exitCode=0 Sep 15 07:56:30.653998 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.653799 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" Sep 15 07:56:30.654079 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.653818 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" event={"ID":"9905d4d3-b112-4d88-9a38-8728d84ab131","Type":"ContainerDied","Data":"1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df"} Sep 15 07:56:30.654297 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.654268 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49" event={"ID":"9905d4d3-b112-4d88-9a38-8728d84ab131","Type":"ContainerDied","Data":"2cc5293ded41629f63a2b78a9b7391a0e8b2b3a4d8aa5d65d0f2b7294d7054f7"} Sep 15 07:56:30.654390 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.654315 2552 scope.go:117] "RemoveContainer" containerID="1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df" Sep 15 07:56:30.663642 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.663624 2552 scope.go:117] "RemoveContainer" containerID="1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df" Sep 15 07:56:30.663938 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:56:30.663917 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df\": container with ID starting with 1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df not found: ID does not exist" containerID="1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df" Sep 15 07:56:30.663994 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.663947 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df"} err="failed to get container status \"1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df\": rpc error: code = NotFound desc = could not find container \"1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df\": container with ID starting with 1b708199861d39de3f10ef388460eae7df543f134e05ed5f6b93fd1ab75a68df not found: ID does not exist" Sep 15 07:56:30.675188 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.675155 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49"] Sep 15 07:56:30.680989 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.680966 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-0fd0de-5cc77969bf-xtw49"] Sep 15 07:56:30.694073 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.694046 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:30.814740 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:30.814708 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:31.474558 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:31.474530 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9905d4d3-b112-4d88-9a38-8728d84ab131" path="/var/lib/kubelet/pods/9905d4d3-b112-4d88-9a38-8728d84ab131/volumes" Sep 15 07:56:31.738311 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:31.738223 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:32.554518 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:32.554482 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:33.233319 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:33.233287 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:34.057829 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:34.057795 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:36.638189 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:36.638153 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:36.874772 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:36.874734 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc"] Sep 15 07:56:36.875040 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:36.875016 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerName="istio-proxy" containerID="cri-o://e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d" gracePeriod=30 Sep 15 07:56:39.636945 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:39.636915 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:39.736897 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:39.736850 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:39.828318 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:39.828286 2552 scope.go:117] "RemoveContainer" containerID="2463a77eefea2f4dc9dd075631dcad04c25daa5c00b922c1f52b3d4c78f38882" Sep 15 07:56:39.836709 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:39.836691 2552 scope.go:117] "RemoveContainer" containerID="e23e5bd2db7b5df7310317fa758271cf4d16e07c88b02ed8051363dc6c81f97c" Sep 15 07:56:40.599389 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:40.599344 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:41.544266 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:41.544236 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:42.130503 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.130480 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:56:42.225265 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.225239 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-envoy\") pod \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " Sep 15 07:56:42.225411 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.225276 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istiod-ca-cert\") pod \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " Sep 15 07:56:42.225411 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.225312 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-workload-certs\") pod \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " Sep 15 07:56:42.225411 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.225339 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-token\") pod \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " Sep 15 07:56:42.225411 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.225371 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-credential-socket\") pod \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " Sep 15 07:56:42.225605 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.225430 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qgf5\" (UniqueName: \"kubernetes.io/projected/93ec2b06-ade3-4ea1-9656-e85fa32022c9-kube-api-access-4qgf5\") pod \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " Sep 15 07:56:42.225605 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.225538 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-podinfo\") pod \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " Sep 15 07:56:42.225605 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.225585 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-workload-socket\") pod \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " Sep 15 07:56:42.225829 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.225609 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-data\") pod \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\" (UID: \"93ec2b06-ade3-4ea1-9656-e85fa32022c9\") " Sep 15 07:56:42.225829 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.225620 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93ec2b06-ade3-4ea1-9656-e85fa32022c9" (UID: "93ec2b06-ade3-4ea1-9656-e85fa32022c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:56:42.225829 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.225767 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93ec2b06-ade3-4ea1-9656-e85fa32022c9" (UID: "93ec2b06-ade3-4ea1-9656-e85fa32022c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:56:42.225829 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.225776 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93ec2b06-ade3-4ea1-9656-e85fa32022c9" (UID: "93ec2b06-ade3-4ea1-9656-e85fa32022c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:56:42.226047 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.226006 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istiod-ca-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:56:42.226047 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.226027 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-workload-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:56:42.226047 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.226035 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-data" (OuterVolumeSpecName: "istio-data") pod "93ec2b06-ade3-4ea1-9656-e85fa32022c9" (UID: "93ec2b06-ade3-4ea1-9656-e85fa32022c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:56:42.226157 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.226040 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-credential-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:56:42.226157 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.226094 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93ec2b06-ade3-4ea1-9656-e85fa32022c9" (UID: "93ec2b06-ade3-4ea1-9656-e85fa32022c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:56:42.227573 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.227543 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93ec2b06-ade3-4ea1-9656-e85fa32022c9" (UID: "93ec2b06-ade3-4ea1-9656-e85fa32022c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:56:42.227678 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.227597 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "93ec2b06-ade3-4ea1-9656-e85fa32022c9" (UID: "93ec2b06-ade3-4ea1-9656-e85fa32022c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:56:42.228073 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.228055 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93ec2b06-ade3-4ea1-9656-e85fa32022c9" (UID: "93ec2b06-ade3-4ea1-9656-e85fa32022c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 07:56:42.228137 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.228069 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ec2b06-ade3-4ea1-9656-e85fa32022c9-kube-api-access-4qgf5" (OuterVolumeSpecName: "kube-api-access-4qgf5") pod "93ec2b06-ade3-4ea1-9656-e85fa32022c9" (UID: "93ec2b06-ade3-4ea1-9656-e85fa32022c9"). InnerVolumeSpecName "kube-api-access-4qgf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:56:42.327101 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.327060 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-envoy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:56:42.327101 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.327096 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-token\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:56:42.327101 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.327105 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qgf5\" (UniqueName: \"kubernetes.io/projected/93ec2b06-ade3-4ea1-9656-e85fa32022c9-kube-api-access-4qgf5\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:56:42.327101 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.327114 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-podinfo\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:56:42.327365 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.327123 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-workload-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:56:42.327365 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.327131 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93ec2b06-ade3-4ea1-9656-e85fa32022c9-istio-data\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:56:42.458893 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.458844 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:42.717937 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.717826 2552 generic.go:356] "Generic (PLEG): container finished" podID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerID="e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d" exitCode=0 Sep 15 07:56:42.718380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.717952 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" Sep 15 07:56:42.718380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.717958 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" event={"ID":"93ec2b06-ade3-4ea1-9656-e85fa32022c9","Type":"ContainerDied","Data":"e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d"} Sep 15 07:56:42.718380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.718024 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc" event={"ID":"93ec2b06-ade3-4ea1-9656-e85fa32022c9","Type":"ContainerDied","Data":"168a609b8c93891665653d6612fcef4286f0815b679c5743252d6c8b85bbebfc"} Sep 15 07:56:42.718380 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.718061 2552 scope.go:117] "RemoveContainer" containerID="e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d" Sep 15 07:56:42.728463 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.728445 2552 scope.go:117] "RemoveContainer" containerID="08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa" Sep 15 07:56:42.736372 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.736356 2552 scope.go:117] "RemoveContainer" containerID="e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d" Sep 15 07:56:42.736656 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:56:42.736632 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d\": container with ID starting with e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d not found: ID does not exist" containerID="e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d" Sep 15 07:56:42.736728 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.736668 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d"} err="failed to get container status \"e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d\": rpc error: code = NotFound desc = could not find container \"e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d\": container with ID starting with e9c2ecd10fe4843ecd4208de8e517600f1efc82a05cc31e858caf1c104e4b87d not found: ID does not exist" Sep 15 07:56:42.736728 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.736694 2552 scope.go:117] "RemoveContainer" containerID="08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa" Sep 15 07:56:42.737008 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:56:42.736990 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa\": container with ID starting with 08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa not found: ID does not exist" containerID="08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa" Sep 15 07:56:42.737060 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.737014 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa"} err="failed to get container status \"08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa\": rpc error: code = NotFound desc = could not find container \"08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa\": container with ID starting with 08ba41a5d185ec7bb4738546f304a7021a94a5fea7b8ca79c76091301f420caa not found: ID does not exist" Sep 15 07:56:42.740160 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.740139 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc"] Sep 15 07:56:42.746245 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:42.746223 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0fd0de-openshift-default-5c447c4d78-xwtmc"] Sep 15 07:56:43.149954 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:43.149924 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:43.475074 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:43.475044 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" path="/var/lib/kubelet/pods/93ec2b06-ade3-4ea1-9656-e85fa32022c9/volumes" Sep 15 07:56:43.950828 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:43.950733 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:44.643330 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:44.643292 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:56.016635 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:56.016605 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:56.140284 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:56.140233 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:56.230477 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:56.230449 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:56.414846 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:56.414759 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:56.583479 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:56.583447 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:56.683819 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:56.683736 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:57.408729 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:57.408691 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:58.349240 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:58.349205 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:56:59.037353 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:56:59.037313 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:01.657177 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:01.657139 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:13.324090 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:13.324039 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:34.128443 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:34.128359 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:37.803512 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:37.803477 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:38.454345 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:38.454309 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:39.868619 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:39.868590 2552 scope.go:117] "RemoveContainer" containerID="9c587c651e424af92f0b78c6f33e38be2ef69717daf7ab87c5aab77283d359b5" Sep 15 07:57:44.041482 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:44.041450 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:44.140057 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:44.140020 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:44.314418 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:44.314335 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:45.050303 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:45.050274 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:45.850275 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:45.850244 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:46.548584 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:46.548556 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:47.321530 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:47.321493 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:48.067422 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:48.067388 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:55.417222 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:55.417192 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:55.482100 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:55.482070 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:55.544214 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:55.544169 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:55.620209 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:55.620178 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:55.685581 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:55.685504 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:55.943619 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:55.943526 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:56.048395 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:56.048365 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:56.706290 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:56.706257 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:57:57.441027 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:57:57.440989 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:01.866213 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:01.866180 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:01.967206 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:01.967174 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:02.134341 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:02.134258 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:02.747232 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:02.747202 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:03.627615 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:03.627577 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:04.344792 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:04.344755 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:05.025470 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:05.025430 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:13.207128 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:13.207095 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5"] Sep 15 07:58:13.207561 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:13.207323 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" podUID="6685c800-6252-4098-82c4-ae1f7419e312" containerName="payload-processing" containerID="cri-o://3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98" gracePeriod=30 Sep 15 07:58:13.450971 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:13.450946 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" Sep 15 07:58:13.571017 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:13.570988 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6685c800-6252-4098-82c4-ae1f7419e312-plugins-config-volume\") pod \"6685c800-6252-4098-82c4-ae1f7419e312\" (UID: \"6685c800-6252-4098-82c4-ae1f7419e312\") " Sep 15 07:58:13.571189 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:13.571176 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7qxw\" (UniqueName: \"kubernetes.io/projected/6685c800-6252-4098-82c4-ae1f7419e312-kube-api-access-g7qxw\") pod \"6685c800-6252-4098-82c4-ae1f7419e312\" (UID: \"6685c800-6252-4098-82c4-ae1f7419e312\") " Sep 15 07:58:13.571404 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:13.571382 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6685c800-6252-4098-82c4-ae1f7419e312-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6685c800-6252-4098-82c4-ae1f7419e312" (UID: "6685c800-6252-4098-82c4-ae1f7419e312"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:58:13.571589 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:13.571573 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6685c800-6252-4098-82c4-ae1f7419e312-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:13.573184 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:13.573157 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6685c800-6252-4098-82c4-ae1f7419e312-kube-api-access-g7qxw" (OuterVolumeSpecName: "kube-api-access-g7qxw") pod "6685c800-6252-4098-82c4-ae1f7419e312" (UID: "6685c800-6252-4098-82c4-ae1f7419e312"). InnerVolumeSpecName "kube-api-access-g7qxw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:58:13.673077 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:13.673051 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7qxw\" (UniqueName: \"kubernetes.io/projected/6685c800-6252-4098-82c4-ae1f7419e312-kube-api-access-g7qxw\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:14.163584 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:14.163543 2552 generic.go:356] "Generic (PLEG): container finished" podID="6685c800-6252-4098-82c4-ae1f7419e312" containerID="3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98" exitCode=0 Sep 15 07:58:14.163785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:14.163608 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" Sep 15 07:58:14.163785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:14.163637 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" event={"ID":"6685c800-6252-4098-82c4-ae1f7419e312","Type":"ContainerDied","Data":"3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98"} Sep 15 07:58:14.163785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:14.163715 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5" event={"ID":"6685c800-6252-4098-82c4-ae1f7419e312","Type":"ContainerDied","Data":"705852c62a52697fedd723e247b12b1fc310dfffc3e30846ebd1ac6952e6b8a5"} Sep 15 07:58:14.163785 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:14.163746 2552 scope.go:117] "RemoveContainer" containerID="3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98" Sep 15 07:58:14.178243 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:14.178209 2552 scope.go:117] "RemoveContainer" containerID="3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98" Sep 15 07:58:14.178565 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:58:14.178544 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98\": container with ID starting with 3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98 not found: ID does not exist" containerID="3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98" Sep 15 07:58:14.178633 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:14.178573 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98"} err="failed to get container status \"3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98\": rpc error: code = NotFound desc = could not find container \"3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98\": container with ID starting with 3161e35048ec909a7ddc9420aaca69c99eb20792ebb152578ad6f551a3ab6b98 not found: ID does not exist" Sep 15 07:58:14.190617 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:14.190593 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5"] Sep 15 07:58:14.192980 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:14.192960 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-04464b-6db6d65595-s9wc5"] Sep 15 07:58:15.475278 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:15.475243 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6685c800-6252-4098-82c4-ae1f7419e312" path="/var/lib/kubelet/pods/6685c800-6252-4098-82c4-ae1f7419e312/volumes" Sep 15 07:58:15.537172 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:15.537139 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:15.598344 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:15.598312 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:19.553655 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:19.553620 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:19.653351 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:19.653317 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:19.789938 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:19.789899 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr"] Sep 15 07:58:19.790215 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:19.790164 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" containerID="cri-o://014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a" gracePeriod=30 Sep 15 07:58:19.814550 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:19.814488 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:20.844881 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:20.844842 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:21.556068 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:21.556033 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:25.039108 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.039074 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:58:25.087081 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087004 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-workload-certs\") pod \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " Sep 15 07:58:25.087081 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087048 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-workload-socket\") pod \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " Sep 15 07:58:25.087081 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087075 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-data\") pod \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " Sep 15 07:58:25.087354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087108 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-podinfo\") pod \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " Sep 15 07:58:25.087354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087133 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-credential-socket\") pod \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " Sep 15 07:58:25.087354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087179 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-envoy\") pod \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " Sep 15 07:58:25.087354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087240 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-token\") pod \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " Sep 15 07:58:25.087354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087274 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istiod-ca-cert\") pod \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " Sep 15 07:58:25.087354 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087322 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwhrl\" (UniqueName: \"kubernetes.io/projected/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-kube-api-access-vwhrl\") pod \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\" (UID: \"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0\") " Sep 15 07:58:25.087667 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087355 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" (UID: "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:58:25.087667 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087367 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" (UID: "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:58:25.087667 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087385 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" (UID: "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:58:25.087840 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087668 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" (UID: "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:58:25.087840 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087724 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-workload-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:25.087840 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087768 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-workload-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:25.087840 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087782 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-data\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:25.087840 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.087726 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" (UID: "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:58:25.089606 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.089581 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" (UID: "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 07:58:25.090006 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.089983 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" (UID: "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:58:25.090108 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.089984 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" (UID: "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:58:25.090157 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.090146 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-kube-api-access-vwhrl" (OuterVolumeSpecName: "kube-api-access-vwhrl") pod "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" (UID: "dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0"). InnerVolumeSpecName "kube-api-access-vwhrl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:58:25.188932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.188903 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-token\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:25.188932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.188926 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istiod-ca-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:25.188932 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.188935 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwhrl\" (UniqueName: \"kubernetes.io/projected/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-kube-api-access-vwhrl\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:25.189140 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.188946 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-podinfo\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:25.189140 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.188955 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-credential-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:25.189140 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.188964 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0-istio-envoy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:25.220091 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.220062 2552 generic.go:356] "Generic (PLEG): container finished" podID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerID="014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a" exitCode=0 Sep 15 07:58:25.220202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.220112 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" Sep 15 07:58:25.220202 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.220157 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" event={"ID":"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0","Type":"ContainerDied","Data":"014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a"} Sep 15 07:58:25.220271 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.220214 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr" event={"ID":"dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0","Type":"ContainerDied","Data":"c5cc37b70082536e99c3f076f8ffc3fe8c833dec9ac0ae4d551ee9822f0b8823"} Sep 15 07:58:25.220271 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.220241 2552 scope.go:117] "RemoveContainer" containerID="014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a" Sep 15 07:58:25.231152 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.231129 2552 scope.go:117] "RemoveContainer" containerID="0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5" Sep 15 07:58:25.240361 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.240346 2552 scope.go:117] "RemoveContainer" containerID="014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a" Sep 15 07:58:25.240615 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:58:25.240585 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a\": container with ID starting with 014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a not found: ID does not exist" containerID="014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a" Sep 15 07:58:25.240700 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.240614 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a"} err="failed to get container status \"014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a\": rpc error: code = NotFound desc = could not find container \"014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a\": container with ID starting with 014daa27d81a907b94d8fb249707b389153a7845a2c7156b823b9d38ea6e852a not found: ID does not exist" Sep 15 07:58:25.240700 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.240631 2552 scope.go:117] "RemoveContainer" containerID="0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5" Sep 15 07:58:25.240880 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:58:25.240848 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5\": container with ID starting with 0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5 not found: ID does not exist" containerID="0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5" Sep 15 07:58:25.240927 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.240887 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5"} err="failed to get container status \"0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5\": rpc error: code = NotFound desc = could not find container \"0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5\": container with ID starting with 0699da4a3bcbd09bc355a99403d16196a3a8f3741a22bfbc660a4a4d07eeafa5 not found: ID does not exist" Sep 15 07:58:25.250056 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.250034 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr"] Sep 15 07:58:25.254142 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.254119 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-04464b-openshift-default-cb67865d8-924nr"] Sep 15 07:58:25.474001 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:25.473981 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" path="/var/lib/kubelet/pods/dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0/volumes" Sep 15 07:58:30.744235 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:30.744200 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:30.745185 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:30.745160 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5"] Sep 15 07:58:30.747939 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:30.747918 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-e3a482-29824305-n9xp5"] Sep 15 07:58:30.781332 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:30.781308 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw"] Sep 15 07:58:30.795335 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:30.795272 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" podUID="f0566e78-fce4-4725-b1e3-dafbe7ea164f" containerName="payload-processing" containerID="cri-o://c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2" gracePeriod=30 Sep 15 07:58:31.059412 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.059381 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" Sep 15 07:58:31.140631 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.140600 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmcjk\" (UniqueName: \"kubernetes.io/projected/f0566e78-fce4-4725-b1e3-dafbe7ea164f-kube-api-access-xmcjk\") pod \"f0566e78-fce4-4725-b1e3-dafbe7ea164f\" (UID: \"f0566e78-fce4-4725-b1e3-dafbe7ea164f\") " Sep 15 07:58:31.140846 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.140659 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0566e78-fce4-4725-b1e3-dafbe7ea164f-plugins-config-volume\") pod \"f0566e78-fce4-4725-b1e3-dafbe7ea164f\" (UID: \"f0566e78-fce4-4725-b1e3-dafbe7ea164f\") " Sep 15 07:58:31.141584 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.141513 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0566e78-fce4-4725-b1e3-dafbe7ea164f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0566e78-fce4-4725-b1e3-dafbe7ea164f" (UID: "f0566e78-fce4-4725-b1e3-dafbe7ea164f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:58:31.142948 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.142918 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0566e78-fce4-4725-b1e3-dafbe7ea164f-kube-api-access-xmcjk" (OuterVolumeSpecName: "kube-api-access-xmcjk") pod "f0566e78-fce4-4725-b1e3-dafbe7ea164f" (UID: "f0566e78-fce4-4725-b1e3-dafbe7ea164f"). InnerVolumeSpecName "kube-api-access-xmcjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:58:31.242076 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.242040 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmcjk\" (UniqueName: \"kubernetes.io/projected/f0566e78-fce4-4725-b1e3-dafbe7ea164f-kube-api-access-xmcjk\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:31.242076 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.242070 2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0566e78-fce4-4725-b1e3-dafbe7ea164f-plugins-config-volume\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:31.251292 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.251266 2552 generic.go:356] "Generic (PLEG): container finished" podID="f0566e78-fce4-4725-b1e3-dafbe7ea164f" containerID="c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2" exitCode=0 Sep 15 07:58:31.251430 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.251312 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" Sep 15 07:58:31.251430 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.251344 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" event={"ID":"f0566e78-fce4-4725-b1e3-dafbe7ea164f","Type":"ContainerDied","Data":"c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2"} Sep 15 07:58:31.251430 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.251421 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw" event={"ID":"f0566e78-fce4-4725-b1e3-dafbe7ea164f","Type":"ContainerDied","Data":"2ce79f596325cd35b0a39d92d822be1133cb3079e3f1cf6c20c193acf19e1f0c"} Sep 15 07:58:31.251538 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.251445 2552 scope.go:117] "RemoveContainer" containerID="c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2" Sep 15 07:58:31.260182 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.260163 2552 scope.go:117] "RemoveContainer" containerID="c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2" Sep 15 07:58:31.260473 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:58:31.260452 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2\": container with ID starting with c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2 not found: ID does not exist" containerID="c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2" Sep 15 07:58:31.260538 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.260485 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2"} err="failed to get container status \"c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2\": rpc error: code = NotFound desc = could not find container \"c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2\": container with ID starting with c3ad01ad03b33dd172dc145d3a297742fc6131a35949a932d2b3ac100ad260d2 not found: ID does not exist" Sep 15 07:58:31.273141 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.273119 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw"] Sep 15 07:58:31.277682 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.277661 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-e3a482-64b846b799-g46dw"] Sep 15 07:58:31.474570 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.474543 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45a4a587-309f-4e9c-91df-5137e35b3f91" path="/var/lib/kubelet/pods/45a4a587-309f-4e9c-91df-5137e35b3f91/volumes" Sep 15 07:58:31.474918 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:31.474905 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0566e78-fce4-4725-b1e3-dafbe7ea164f" path="/var/lib/kubelet/pods/f0566e78-fce4-4725-b1e3-dafbe7ea164f/volumes" Sep 15 07:58:36.027219 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:36.027181 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:36.999928 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:36.999888 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:37.094459 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:37.094427 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:37.243226 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:37.243189 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6"] Sep 15 07:58:37.243470 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:37.243446 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerName="istio-proxy" containerID="cri-o://3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758" gracePeriod=30 Sep 15 07:58:37.265416 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:37.265351 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:38.644001 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:38.643967 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:38.744622 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:38.744591 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 07:58:39.890827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:39.890797 2552 scope.go:117] "RemoveContainer" containerID="a7e3797bc46b5b0022ea4856a3fd7bae286f37e743a97e757f6a39ffba6ce297" Sep 15 07:58:41.176885 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:41.176848 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:58:41.183854 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:41.183832 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:58:41.186049 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:41.186022 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 07:58:41.192883 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:41.192849 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 07:58:42.499216 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.499190 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:58:42.545206 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545125 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-token\") pod \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " Sep 15 07:58:42.545206 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545190 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-data\") pod \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " Sep 15 07:58:42.545431 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545228 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-workload-certs\") pod \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " Sep 15 07:58:42.545431 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545246 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-workload-socket\") pod \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " Sep 15 07:58:42.545431 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545286 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-envoy\") pod \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " Sep 15 07:58:42.545431 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545331 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-credential-socket\") pod \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " Sep 15 07:58:42.545431 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545381 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qq7jw\" (UniqueName: \"kubernetes.io/projected/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-kube-api-access-qq7jw\") pod \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " Sep 15 07:58:42.545431 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545419 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istiod-ca-cert\") pod \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " Sep 15 07:58:42.545738 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545448 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-podinfo\") pod \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\" (UID: \"03e01a9e-b2d5-4ee9-97c5-157c8d181d32\") " Sep 15 07:58:42.545738 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545484 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-data" (OuterVolumeSpecName: "istio-data") pod "03e01a9e-b2d5-4ee9-97c5-157c8d181d32" (UID: "03e01a9e-b2d5-4ee9-97c5-157c8d181d32"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:58:42.545738 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545645 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "03e01a9e-b2d5-4ee9-97c5-157c8d181d32" (UID: "03e01a9e-b2d5-4ee9-97c5-157c8d181d32"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:58:42.545738 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545661 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "03e01a9e-b2d5-4ee9-97c5-157c8d181d32" (UID: "03e01a9e-b2d5-4ee9-97c5-157c8d181d32"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:58:42.545989 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545824 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "03e01a9e-b2d5-4ee9-97c5-157c8d181d32" (UID: "03e01a9e-b2d5-4ee9-97c5-157c8d181d32"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:58:42.545989 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545879 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "03e01a9e-b2d5-4ee9-97c5-157c8d181d32" (UID: "03e01a9e-b2d5-4ee9-97c5-157c8d181d32"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:58:42.545989 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545889 2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-data\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:42.545989 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545910 2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-workload-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:42.545989 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.545923 2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-credential-socket\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:42.547503 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.547477 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-kube-api-access-qq7jw" (OuterVolumeSpecName: "kube-api-access-qq7jw") pod "03e01a9e-b2d5-4ee9-97c5-157c8d181d32" (UID: "03e01a9e-b2d5-4ee9-97c5-157c8d181d32"). InnerVolumeSpecName "kube-api-access-qq7jw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:58:42.547783 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.547765 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-token" (OuterVolumeSpecName: "istio-token") pod "03e01a9e-b2d5-4ee9-97c5-157c8d181d32" (UID: "03e01a9e-b2d5-4ee9-97c5-157c8d181d32"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:58:42.547940 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.547922 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "03e01a9e-b2d5-4ee9-97c5-157c8d181d32" (UID: "03e01a9e-b2d5-4ee9-97c5-157c8d181d32"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 15 07:58:42.547996 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.547935 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "03e01a9e-b2d5-4ee9-97c5-157c8d181d32" (UID: "03e01a9e-b2d5-4ee9-97c5-157c8d181d32"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:58:42.647309 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.647268 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq7jw\" (UniqueName: \"kubernetes.io/projected/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-kube-api-access-qq7jw\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:42.647309 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.647302 2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istiod-ca-cert\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:42.647309 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.647313 2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-podinfo\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:42.647537 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.647321 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-token\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:42.647537 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.647331 2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-workload-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:42.647537 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:42.647338 2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03e01a9e-b2d5-4ee9-97c5-157c8d181d32-istio-envoy\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 07:58:43.312641 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.312603 2552 generic.go:356] "Generic (PLEG): container finished" podID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerID="3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758" exitCode=0 Sep 15 07:58:43.312827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.312656 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" Sep 15 07:58:43.312827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.312687 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" event={"ID":"03e01a9e-b2d5-4ee9-97c5-157c8d181d32","Type":"ContainerDied","Data":"3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758"} Sep 15 07:58:43.312827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.312739 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6" event={"ID":"03e01a9e-b2d5-4ee9-97c5-157c8d181d32","Type":"ContainerDied","Data":"4c452f7fcefc953426a621c2f7c79bf61e3a7658e37c2ea7b5490279ccb21938"} Sep 15 07:58:43.312827 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.312770 2552 scope.go:117] "RemoveContainer" containerID="3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758" Sep 15 07:58:43.322223 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.322204 2552 scope.go:117] "RemoveContainer" containerID="957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf" Sep 15 07:58:43.329918 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.329899 2552 scope.go:117] "RemoveContainer" containerID="3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758" Sep 15 07:58:43.330238 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:58:43.330218 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758\": container with ID starting with 3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758 not found: ID does not exist" containerID="3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758" Sep 15 07:58:43.330328 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.330244 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758"} err="failed to get container status \"3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758\": rpc error: code = NotFound desc = could not find container \"3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758\": container with ID starting with 3044fddca3a3d08c01726b46ce7de2f8676723750ada3bcbb123d21f97b63758 not found: ID does not exist" Sep 15 07:58:43.330328 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.330260 2552 scope.go:117] "RemoveContainer" containerID="957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf" Sep 15 07:58:43.330499 ip-10-0-143-175 kubenswrapper[2552]: E0915 07:58:43.330485 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf\": container with ID starting with 957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf not found: ID does not exist" containerID="957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf" Sep 15 07:58:43.330538 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.330502 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf"} err="failed to get container status \"957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf\": rpc error: code = NotFound desc = could not find container \"957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf\": container with ID starting with 957f0aa1bc5ead94cf8e0ca7d2fecf08f950eb54c34918e1eb4f82cc6ce544cf not found: ID does not exist" Sep 15 07:58:43.333397 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.333373 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6"] Sep 15 07:58:43.338059 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.337855 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e3a482-openshift-default-85598c95bd-kbwg6"] Sep 15 07:58:43.474243 ip-10-0-143-175 kubenswrapper[2552]: I0915 07:58:43.474213 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" path="/var/lib/kubelet/pods/03e01a9e-b2d5-4ee9-97c5-157c8d181d32/volumes" Sep 15 08:00:03.247691 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:00:03.247651 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:00:04.671527 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:00:04.671474 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:00:40.667421 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:00:40.667375 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:01:42.650609 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:01:42.650569 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:02:07.244843 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:02:07.244785 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:02:32.336491 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:02:32.336458 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:02:47.198799 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:02:47.198762 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:02:51.927147 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:02:51.927100 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:03:41.203794 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:03:41.203765 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 08:03:41.210584 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:03:41.210560 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 08:03:41.215285 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:03:41.215266 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 08:03:41.221640 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:03:41.221623 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkh69_686793e7-d253-4516-9e2a-e53f8f528cf4/ovn-acl-logging/0.log" Sep 15 08:03:46.586104 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:03:46.586069 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:03:49.237700 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:03:49.237670 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:03:49.339563 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:03:49.339527 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:04:09.633999 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.633952 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq"] Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634292 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerName="istio-proxy" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634303 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerName="istio-proxy" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634316 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634321 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634328 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0566e78-fce4-4725-b1e3-dafbe7ea164f" containerName="payload-processing" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634334 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0566e78-fce4-4725-b1e3-dafbe7ea164f" containerName="payload-processing" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634350 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9905d4d3-b112-4d88-9a38-8728d84ab131" containerName="payload-processing" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634356 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9905d4d3-b112-4d88-9a38-8728d84ab131" containerName="payload-processing" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634363 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6685c800-6252-4098-82c4-ae1f7419e312" containerName="payload-processing" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634368 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6685c800-6252-4098-82c4-ae1f7419e312" containerName="payload-processing" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634374 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerName="istio-proxy" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634379 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerName="istio-proxy" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634387 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerName="istio-proxy" Sep 15 08:04:09.634408 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634392 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerName="istio-proxy" Sep 15 08:04:09.634841 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634444 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0566e78-fce4-4725-b1e3-dafbe7ea164f" containerName="payload-processing" Sep 15 08:04:09.634841 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634451 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerName="istio-proxy" Sep 15 08:04:09.634841 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634457 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="9905d4d3-b112-4d88-9a38-8728d84ab131" containerName="payload-processing" Sep 15 08:04:09.634841 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634462 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="03e01a9e-b2d5-4ee9-97c5-157c8d181d32" containerName="istio-proxy" Sep 15 08:04:09.634841 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634469 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerName="istio-proxy" Sep 15 08:04:09.634841 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634473 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="6685c800-6252-4098-82c4-ae1f7419e312" containerName="payload-processing" Sep 15 08:04:09.634841 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634478 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerName="istio-proxy" Sep 15 08:04:09.634841 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.634484 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" Sep 15 08:04:09.637442 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.637415 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.640583 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.640559 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xf69m\"" Sep 15 08:04:09.640583 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.640580 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 15 08:04:09.640821 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.640600 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 15 08:04:09.640821 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.640604 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 15 08:04:09.640821 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.640711 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 15 08:04:09.651346 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.651319 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq"] Sep 15 08:04:09.733998 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.733965 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77bbafcf-2603-4be5-b773-80764fe172ce-metrics-certs\") pod \"maas-api-66bb9979b8-dlrmq\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.733998 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.734001 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxjkg\" (UniqueName: \"kubernetes.io/projected/77bbafcf-2603-4be5-b773-80764fe172ce-kube-api-access-lxjkg\") pod \"maas-api-66bb9979b8-dlrmq\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.734205 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.734118 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77bbafcf-2603-4be5-b773-80764fe172ce-maas-api-tls\") pod \"maas-api-66bb9979b8-dlrmq\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.734205 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.734172 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77bbafcf-2603-4be5-b773-80764fe172ce-openshift-service-ca\") pod \"maas-api-66bb9979b8-dlrmq\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.835511 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.835481 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77bbafcf-2603-4be5-b773-80764fe172ce-maas-api-tls\") pod \"maas-api-66bb9979b8-dlrmq\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.835687 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.835526 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77bbafcf-2603-4be5-b773-80764fe172ce-openshift-service-ca\") pod \"maas-api-66bb9979b8-dlrmq\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.835687 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.835558 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77bbafcf-2603-4be5-b773-80764fe172ce-metrics-certs\") pod \"maas-api-66bb9979b8-dlrmq\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.835687 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.835584 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxjkg\" (UniqueName: \"kubernetes.io/projected/77bbafcf-2603-4be5-b773-80764fe172ce-kube-api-access-lxjkg\") pod \"maas-api-66bb9979b8-dlrmq\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.836272 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.836249 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77bbafcf-2603-4be5-b773-80764fe172ce-openshift-service-ca\") pod \"maas-api-66bb9979b8-dlrmq\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.837919 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.837897 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77bbafcf-2603-4be5-b773-80764fe172ce-maas-api-tls\") pod \"maas-api-66bb9979b8-dlrmq\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.838010 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.837940 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77bbafcf-2603-4be5-b773-80764fe172ce-metrics-certs\") pod \"maas-api-66bb9979b8-dlrmq\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.843738 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.843717 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxjkg\" (UniqueName: \"kubernetes.io/projected/77bbafcf-2603-4be5-b773-80764fe172ce-kube-api-access-lxjkg\") pod \"maas-api-66bb9979b8-dlrmq\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:09.950637 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:09.950545 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:10.223445 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:10.223371 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq"] Sep 15 08:04:10.226459 ip-10-0-143-175 kubenswrapper[2552]: W0915 08:04:10.226430 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77bbafcf_2603_4be5_b773_80764fe172ce.slice/crio-b079eb0c5aa6e95d5217f99c27f0c8d8cab1dd0f2cdbf7a8b38e741221cfe53d WatchSource:0}: Error finding container b079eb0c5aa6e95d5217f99c27f0c8d8cab1dd0f2cdbf7a8b38e741221cfe53d: Status 404 returned error can't find the container with id b079eb0c5aa6e95d5217f99c27f0c8d8cab1dd0f2cdbf7a8b38e741221cfe53d Sep 15 08:04:10.227827 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:10.227807 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 08:04:10.824014 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:10.823979 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" event={"ID":"77bbafcf-2603-4be5-b773-80764fe172ce","Type":"ContainerStarted","Data":"b079eb0c5aa6e95d5217f99c27f0c8d8cab1dd0f2cdbf7a8b38e741221cfe53d"} Sep 15 08:04:12.834860 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:12.834821 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" event={"ID":"77bbafcf-2603-4be5-b773-80764fe172ce","Type":"ContainerStarted","Data":"592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039"} Sep 15 08:04:12.835316 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:12.834921 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:12.851681 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:12.851630 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" podStartSLOduration=2.02839655 podStartE2EDuration="3.85161341s" totalImagesPullingTime="1.82321686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 08:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 08:04:12.850662201 +0000 UTC m=+2733.831824378" watchObservedRunningTime="2026-09-15 08:04:12.85161341 +0000 UTC m=+2733.832775577" Sep 15 08:04:17.379803 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.379771 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq"] Sep 15 08:04:17.380263 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.380037 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" podUID="77bbafcf-2603-4be5-b773-80764fe172ce" containerName="maas-api" containerID="cri-o://592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039" gracePeriod=30 Sep 15 08:04:17.385142 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.385119 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:17.418785 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.418760 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b6696b464-xxths"] Sep 15 08:04:17.419145 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.419131 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerName="istio-proxy" Sep 15 08:04:17.419209 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.419147 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ec2b06-ade3-4ea1-9656-e85fa32022c9" containerName="istio-proxy" Sep 15 08:04:17.419209 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.419159 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" Sep 15 08:04:17.419209 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.419170 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" Sep 15 08:04:17.419332 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.419235 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc5e2d6c-ff5c-49e5-b6ff-10fcae0c5cc0" containerName="istio-proxy" Sep 15 08:04:17.422709 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.422689 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.433260 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.433239 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b6696b464-xxths"] Sep 15 08:04:17.515200 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.515169 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7313ddc5-6b0b-4642-b8b7-09d76e7a68ff-maas-api-tls\") pod \"maas-api-6b6696b464-xxths\" (UID: \"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff\") " pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.515388 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.515213 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7313ddc5-6b0b-4642-b8b7-09d76e7a68ff-openshift-service-ca\") pod \"maas-api-6b6696b464-xxths\" (UID: \"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff\") " pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.515388 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.515264 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7313ddc5-6b0b-4642-b8b7-09d76e7a68ff-metrics-certs\") pod \"maas-api-6b6696b464-xxths\" (UID: \"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff\") " pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.515388 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.515349 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj75g\" (UniqueName: \"kubernetes.io/projected/7313ddc5-6b0b-4642-b8b7-09d76e7a68ff-kube-api-access-mj75g\") pod \"maas-api-6b6696b464-xxths\" (UID: \"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff\") " pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.616560 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.616533 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7313ddc5-6b0b-4642-b8b7-09d76e7a68ff-openshift-service-ca\") pod \"maas-api-6b6696b464-xxths\" (UID: \"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff\") " pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.616699 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.616592 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7313ddc5-6b0b-4642-b8b7-09d76e7a68ff-metrics-certs\") pod \"maas-api-6b6696b464-xxths\" (UID: \"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff\") " pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.616699 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.616655 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj75g\" (UniqueName: \"kubernetes.io/projected/7313ddc5-6b0b-4642-b8b7-09d76e7a68ff-kube-api-access-mj75g\") pod \"maas-api-6b6696b464-xxths\" (UID: \"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff\") " pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.616827 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.616757 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7313ddc5-6b0b-4642-b8b7-09d76e7a68ff-maas-api-tls\") pod \"maas-api-6b6696b464-xxths\" (UID: \"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff\") " pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.617410 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.617381 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7313ddc5-6b0b-4642-b8b7-09d76e7a68ff-openshift-service-ca\") pod \"maas-api-6b6696b464-xxths\" (UID: \"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff\") " pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.619622 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.619600 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7313ddc5-6b0b-4642-b8b7-09d76e7a68ff-maas-api-tls\") pod \"maas-api-6b6696b464-xxths\" (UID: \"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff\") " pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.619769 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.619747 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7313ddc5-6b0b-4642-b8b7-09d76e7a68ff-metrics-certs\") pod \"maas-api-6b6696b464-xxths\" (UID: \"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff\") " pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.625341 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.625323 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj75g\" (UniqueName: \"kubernetes.io/projected/7313ddc5-6b0b-4642-b8b7-09d76e7a68ff-kube-api-access-mj75g\") pod \"maas-api-6b6696b464-xxths\" (UID: \"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff\") " pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.638442 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.638396 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:17.760650 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.760611 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:17.822254 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.819408 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxjkg\" (UniqueName: \"kubernetes.io/projected/77bbafcf-2603-4be5-b773-80764fe172ce-kube-api-access-lxjkg\") pod \"77bbafcf-2603-4be5-b773-80764fe172ce\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " Sep 15 08:04:17.822254 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.819458 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77bbafcf-2603-4be5-b773-80764fe172ce-metrics-certs\") pod \"77bbafcf-2603-4be5-b773-80764fe172ce\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " Sep 15 08:04:17.822254 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.819502 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77bbafcf-2603-4be5-b773-80764fe172ce-maas-api-tls\") pod \"77bbafcf-2603-4be5-b773-80764fe172ce\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " Sep 15 08:04:17.822254 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.819638 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77bbafcf-2603-4be5-b773-80764fe172ce-openshift-service-ca\") pod \"77bbafcf-2603-4be5-b773-80764fe172ce\" (UID: \"77bbafcf-2603-4be5-b773-80764fe172ce\") " Sep 15 08:04:17.822254 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.820285 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77bbafcf-2603-4be5-b773-80764fe172ce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "77bbafcf-2603-4be5-b773-80764fe172ce" (UID: "77bbafcf-2603-4be5-b773-80764fe172ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 08:04:17.826905 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.826853 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77bbafcf-2603-4be5-b773-80764fe172ce-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "77bbafcf-2603-4be5-b773-80764fe172ce" (UID: "77bbafcf-2603-4be5-b773-80764fe172ce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 08:04:17.827036 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.826993 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77bbafcf-2603-4be5-b773-80764fe172ce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "77bbafcf-2603-4be5-b773-80764fe172ce" (UID: "77bbafcf-2603-4be5-b773-80764fe172ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 08:04:17.827289 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.827250 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77bbafcf-2603-4be5-b773-80764fe172ce-kube-api-access-lxjkg" (OuterVolumeSpecName: "kube-api-access-lxjkg") pod "77bbafcf-2603-4be5-b773-80764fe172ce" (UID: "77bbafcf-2603-4be5-b773-80764fe172ce"). InnerVolumeSpecName "kube-api-access-lxjkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 08:04:17.858650 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.858621 2552 generic.go:356] "Generic (PLEG): container finished" podID="77bbafcf-2603-4be5-b773-80764fe172ce" containerID="592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039" exitCode=0 Sep 15 08:04:17.858816 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.858672 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" Sep 15 08:04:17.858816 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.858713 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" event={"ID":"77bbafcf-2603-4be5-b773-80764fe172ce","Type":"ContainerDied","Data":"592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039"} Sep 15 08:04:17.858816 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.858804 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq" event={"ID":"77bbafcf-2603-4be5-b773-80764fe172ce","Type":"ContainerDied","Data":"b079eb0c5aa6e95d5217f99c27f0c8d8cab1dd0f2cdbf7a8b38e741221cfe53d"} Sep 15 08:04:17.859018 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.858834 2552 scope.go:117] "RemoveContainer" containerID="592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039" Sep 15 08:04:17.867739 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.867716 2552 scope.go:117] "RemoveContainer" containerID="592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039" Sep 15 08:04:17.868037 ip-10-0-143-175 kubenswrapper[2552]: E0915 08:04:17.868018 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039\": container with ID starting with 592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039 not found: ID does not exist" containerID="592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039" Sep 15 08:04:17.868108 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.868046 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039"} err="failed to get container status \"592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039\": rpc error: code = NotFound desc = could not find container \"592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039\": container with ID starting with 592a0670afe74503755ec8cb4d94b0548116baa14c77050580aea84053b1b039 not found: ID does not exist" Sep 15 08:04:17.880526 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.880503 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq"] Sep 15 08:04:17.884109 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.884087 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-66bb9979b8-dlrmq"] Sep 15 08:04:17.920430 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.920370 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77bbafcf-2603-4be5-b773-80764fe172ce-maas-api-tls\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 08:04:17.920430 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.920388 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77bbafcf-2603-4be5-b773-80764fe172ce-openshift-service-ca\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 08:04:17.920430 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.920402 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxjkg\" (UniqueName: \"kubernetes.io/projected/77bbafcf-2603-4be5-b773-80764fe172ce-kube-api-access-lxjkg\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 08:04:17.920430 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:17.920416 2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77bbafcf-2603-4be5-b773-80764fe172ce-metrics-certs\") on node \"ip-10-0-143-175.ec2.internal\" DevicePath \"\"" Sep 15 08:04:18.031484 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:18.031459 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b6696b464-xxths"] Sep 15 08:04:18.033918 ip-10-0-143-175 kubenswrapper[2552]: W0915 08:04:18.033892 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7313ddc5_6b0b_4642_b8b7_09d76e7a68ff.slice/crio-88c0dda52ba52a5adb79b61b1db539f940080718da80ffb0531de0443bab131e WatchSource:0}: Error finding container 88c0dda52ba52a5adb79b61b1db539f940080718da80ffb0531de0443bab131e: Status 404 returned error can't find the container with id 88c0dda52ba52a5adb79b61b1db539f940080718da80ffb0531de0443bab131e Sep 15 08:04:18.865571 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:18.865533 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" event={"ID":"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff","Type":"ContainerStarted","Data":"92f72e8c85e0ac635cf7749855b7896fe8ebf8c8783c2727cd71170f041793c9"} Sep 15 08:04:18.865959 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:18.865593 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" event={"ID":"7313ddc5-6b0b-4642-b8b7-09d76e7a68ff","Type":"ContainerStarted","Data":"88c0dda52ba52a5adb79b61b1db539f940080718da80ffb0531de0443bab131e"} Sep 15 08:04:18.865959 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:18.865651 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:18.883598 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:18.883558 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" podStartSLOduration=1.437081305 podStartE2EDuration="1.883542813s" totalImagesPullingTime="446.461508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 08:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 08:04:18.881684673 +0000 UTC m=+2739.862846841" watchObservedRunningTime="2026-09-15 08:04:18.883542813 +0000 UTC m=+2739.864704982" Sep 15 08:04:19.473926 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:19.473898 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77bbafcf-2603-4be5-b773-80764fe172ce" path="/var/lib/kubelet/pods/77bbafcf-2603-4be5-b773-80764fe172ce/volumes" Sep 15 08:04:24.153841 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:24.153799 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:04:24.692621 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:24.692582 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:04:24.875917 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:24.875888 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b6696b464-xxths" Sep 15 08:04:45.285548 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:45.285512 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jzhtd_5b648781-f1de-428f-8606-9861f52bb7db/server/0.log" Sep 15 08:04:45.664815 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:45.664745 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-565d545f7b-lpnqs_4e90840b-cab1-407b-8fff-e1feac2c3a58/copy-manifests/0.log" Sep 15 08:04:45.682588 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:45.682563 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-565d545f7b-lpnqs_4e90840b-cab1-407b-8fff-e1feac2c3a58/manager/0.log" Sep 15 08:04:46.273804 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:46.273771 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6b6696b464-xxths_7313ddc5-6b0b-4642-b8b7-09d76e7a68ff/maas-api/0.log" Sep 15 08:04:48.550567 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:48.550536 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt_024f5b7f-74ab-4aaf-9dc9-d9bf9e089707/istio-proxy/0.log" Sep 15 08:04:48.674775 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:48.674739 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-42jct_6091e72d-1db6-4b32-9420-f1cd119c990b/discovery/0.log" Sep 15 08:04:49.265236 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:49.265199 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-cgt48_11660574-efac-4686-962b-934e3bd76287/payload-processing/0.log" Sep 15 08:04:49.382680 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:49.382650 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c5cb56bb8-jq29f_a951ccbf-4363-44e1-9e9b-45a3ae082e7e/router/0.log" Sep 15 08:04:49.778508 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:49.778478 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-592xx_5e0c1ffc-e0ee-4a6f-80b6-7051d0bc36a8/main/0.log" Sep 15 08:04:50.023411 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:50.023382 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-lfrvk_83b47a7d-a312-453b-8ebb-3e6a27b48459/main/0.log" Sep 15 08:04:50.361581 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:50.361550 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-fkbwv_2988082b-93a8-48fd-bdfe-11977c4e016f/main/0.log" Sep 15 08:04:55.611787 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:55.611758 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:04:58.503552 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:58.503515 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" limit=1024 Sep 15 08:04:58.556274 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:58.556249 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h8b5m_c6f6820e-f789-494a-9a66-fe16e0c2f21a/global-pull-secret-syncer/0.log" Sep 15 08:04:58.647437 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:58.647409 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9mt82_4b1e23db-0eff-418c-8f36-3e5dedc822ba/konnectivity-agent/0.log" Sep 15 08:04:58.771723 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:04:58.771629 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-175.ec2.internal_ff4bf43fd141c579255e728a935942fa/haproxy/0.log" Sep 15 08:05:05.493376 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:05.493340 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c377b38a-a1ed-4a58-a11d-7ca077287976/alertmanager/0.log" Sep 15 08:05:05.516960 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:05.516936 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c377b38a-a1ed-4a58-a11d-7ca077287976/config-reloader/0.log" Sep 15 08:05:05.541607 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:05.541584 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c377b38a-a1ed-4a58-a11d-7ca077287976/kube-rbac-proxy-web/0.log" Sep 15 08:05:05.562583 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:05.562563 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c377b38a-a1ed-4a58-a11d-7ca077287976/kube-rbac-proxy/0.log" Sep 15 08:05:05.583603 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:05.583582 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c377b38a-a1ed-4a58-a11d-7ca077287976/kube-rbac-proxy-metric/0.log" Sep 15 08:05:05.604142 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:05.604119 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c377b38a-a1ed-4a58-a11d-7ca077287976/prom-label-proxy/0.log" Sep 15 08:05:05.627722 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:05.627704 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c377b38a-a1ed-4a58-a11d-7ca077287976/init-config-reloader/0.log" Sep 15 08:05:06.019725 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:06.019637 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxcpq_ea4664a6-2ac7-4a77-bd97-91e53c772091/node-exporter/0.log" Sep 15 08:05:06.038154 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:06.038124 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxcpq_ea4664a6-2ac7-4a77-bd97-91e53c772091/kube-rbac-proxy/0.log" Sep 15 08:05:06.057693 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:06.057670 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxcpq_ea4664a6-2ac7-4a77-bd97-91e53c772091/init-textfile/0.log" Sep 15 08:05:06.426506 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:06.426428 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-l42tm_3ad379ee-05df-4ab2-b2b3-302669226c77/prometheus-operator/0.log" Sep 15 08:05:06.445560 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:06.445532 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-l42tm_3ad379ee-05df-4ab2-b2b3-302669226c77/kube-rbac-proxy/0.log" Sep 15 08:05:06.469782 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:06.469758 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-msdwd_2e6011ff-a0de-44c7-8899-d0b57b5a2e88/prometheus-operator-admission-webhook/0.log" Sep 15 08:05:06.503126 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:06.503101 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cf9c55f4d-sklkl_119ff648-9dee-42b1-b951-864f117fad51/telemeter-client/0.log" Sep 15 08:05:06.522616 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:06.522597 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cf9c55f4d-sklkl_119ff648-9dee-42b1-b951-864f117fad51/reload/0.log" Sep 15 08:05:06.543950 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:06.543926 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cf9c55f4d-sklkl_119ff648-9dee-42b1-b951-864f117fad51/kube-rbac-proxy/0.log" Sep 15 08:05:08.248430 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.248400 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x"] Sep 15 08:05:08.248842 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.248729 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77bbafcf-2603-4be5-b773-80764fe172ce" containerName="maas-api" Sep 15 08:05:08.248842 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.248740 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="77bbafcf-2603-4be5-b773-80764fe172ce" containerName="maas-api" Sep 15 08:05:08.248842 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.248815 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="77bbafcf-2603-4be5-b773-80764fe172ce" containerName="maas-api" Sep 15 08:05:08.252129 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.252110 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.254394 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.254372 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c8zpm\"/\"openshift-service-ca.crt\"" Sep 15 08:05:08.254499 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.254432 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c8zpm\"/\"kube-root-ca.crt\"" Sep 15 08:05:08.255168 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.255151 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c8zpm\"/\"default-dockercfg-5fl5x\"" Sep 15 08:05:08.261746 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.261723 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x"] Sep 15 08:05:08.264695 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.264670 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ad1a28b-4845-4b69-bde7-75cf7725ede5-lib-modules\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.264791 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.264708 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8ad1a28b-4845-4b69-bde7-75cf7725ede5-proc\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.264791 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.264765 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8ad1a28b-4845-4b69-bde7-75cf7725ede5-podres\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.264921 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.264795 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ad1a28b-4845-4b69-bde7-75cf7725ede5-sys\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.264921 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.264839 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-754pm\" (UniqueName: \"kubernetes.io/projected/8ad1a28b-4845-4b69-bde7-75cf7725ede5-kube-api-access-754pm\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.365765 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.365736 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8ad1a28b-4845-4b69-bde7-75cf7725ede5-podres\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.365942 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.365788 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ad1a28b-4845-4b69-bde7-75cf7725ede5-sys\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.365942 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.365822 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-754pm\" (UniqueName: \"kubernetes.io/projected/8ad1a28b-4845-4b69-bde7-75cf7725ede5-kube-api-access-754pm\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.365942 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.365848 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ad1a28b-4845-4b69-bde7-75cf7725ede5-lib-modules\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.365942 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.365910 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8ad1a28b-4845-4b69-bde7-75cf7725ede5-proc\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.365942 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.365933 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8ad1a28b-4845-4b69-bde7-75cf7725ede5-podres\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.366145 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.365961 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ad1a28b-4845-4b69-bde7-75cf7725ede5-sys\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.366145 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.365989 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8ad1a28b-4845-4b69-bde7-75cf7725ede5-proc\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.366145 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.366017 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ad1a28b-4845-4b69-bde7-75cf7725ede5-lib-modules\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.373367 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.373348 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-754pm\" (UniqueName: \"kubernetes.io/projected/8ad1a28b-4845-4b69-bde7-75cf7725ede5-kube-api-access-754pm\") pod \"perf-node-gather-daemonset-ndq7x\" (UID: \"8ad1a28b-4845-4b69-bde7-75cf7725ede5\") " pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.560936 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.560823 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/2.log" Sep 15 08:05:08.561616 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.561591 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:08.569032 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.569008 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-cdwxq_a0dd4f12-537f-42ae-99c6-2d900b956860/console-operator/3.log" Sep 15 08:05:08.835853 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:08.835777 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x"] Sep 15 08:05:08.839760 ip-10-0-143-175 kubenswrapper[2552]: W0915 08:05:08.839737 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8ad1a28b_4845_4b69_bde7_75cf7725ede5.slice/crio-db9bd339b6415abb481c9c3499b35bdb3938e83dd5a076ada41f6e300783beac WatchSource:0}: Error finding container db9bd339b6415abb481c9c3499b35bdb3938e83dd5a076ada41f6e300783beac: Status 404 returned error can't find the container with id db9bd339b6415abb481c9c3499b35bdb3938e83dd5a076ada41f6e300783beac Sep 15 08:05:09.105548 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:09.105454 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" event={"ID":"8ad1a28b-4845-4b69-bde7-75cf7725ede5","Type":"ContainerStarted","Data":"73629b9f0f46480c6b1942899c9b8f6a555955518a7531eb65a54d82fc99c776"} Sep 15 08:05:09.105548 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:09.105506 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" event={"ID":"8ad1a28b-4845-4b69-bde7-75cf7725ede5","Type":"ContainerStarted","Data":"db9bd339b6415abb481c9c3499b35bdb3938e83dd5a076ada41f6e300783beac"} Sep 15 08:05:09.105548 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:09.105531 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:09.121510 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:09.121465 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" podStartSLOduration=1.121450973 podStartE2EDuration="1.121450973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 08:05:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 08:05:09.119125302 +0000 UTC m=+2790.100287469" watchObservedRunningTime="2026-09-15 08:05:09.121450973 +0000 UTC m=+2790.102613139" Sep 15 08:05:09.162942 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:09.162914 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-q67fs_e617ad16-f2c5-41a9-b78b-e2936a79f16b/download-server/0.log" Sep 15 08:05:10.888005 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:10.887947 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bsrr6_ee9505e1-41b8-4f77-b211-c41076a8eac9/dns/0.log" Sep 15 08:05:10.907539 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:10.907511 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bsrr6_ee9505e1-41b8-4f77-b211-c41076a8eac9/kube-rbac-proxy/0.log" Sep 15 08:05:11.042982 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:11.042949 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xbtjp_42649c54-356b-4265-9d26-2c42d9b0f265/dns-node-resolver/0.log" Sep 15 08:05:11.651220 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:11.651197 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-njnqn_7e2134f2-9ba8-432a-9ba9-aa8e313a86c6/node-ca/0.log" Sep 15 08:05:12.597267 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:12.597235 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt_024f5b7f-74ab-4aaf-9dc9-d9bf9e089707/istio-proxy/0.log" Sep 15 08:05:13.409927 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:13.409893 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-42jct_6091e72d-1db6-4b32-9420-f1cd119c990b/discovery/0.log" Sep 15 08:05:13.554510 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:13.554484 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-cgt48_11660574-efac-4686-962b-934e3bd76287/payload-processing/0.log" Sep 15 08:05:13.576582 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:13.576545 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c5cb56bb8-jq29f_a951ccbf-4363-44e1-9e9b-45a3ae082e7e/router/0.log" Sep 15 08:05:14.497030 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:14.496998 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bdtkq_4169ca54-51ee-4dc7-9280-649b5143d3bf/serve-healthcheck-canary/0.log" Sep 15 08:05:14.944062 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:14.943975 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-46pvm_449e530a-1a2b-4707-8272-48ec5851253d/insights-operator/0.log" Sep 15 08:05:14.944903 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:14.944842 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-46pvm_449e530a-1a2b-4707-8272-48ec5851253d/insights-operator/1.log" Sep 15 08:05:14.967901 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:14.967860 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cttzt_5ef49b13-2791-4cf4-9de5-8ac40eae2eee/kube-rbac-proxy/0.log" Sep 15 08:05:14.990198 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:14.990175 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cttzt_5ef49b13-2791-4cf4-9de5-8ac40eae2eee/exporter/0.log" Sep 15 08:05:15.013368 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:15.013345 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cttzt_5ef49b13-2791-4cf4-9de5-8ac40eae2eee/extractor/0.log" Sep 15 08:05:15.120808 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:15.120777 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c8zpm/perf-node-gather-daemonset-ndq7x" Sep 15 08:05:17.906898 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:17.906855 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jzhtd_5b648781-f1de-428f-8606-9861f52bb7db/server/0.log" Sep 15 08:05:18.193070 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:18.192982 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-565d545f7b-lpnqs_4e90840b-cab1-407b-8fff-e1feac2c3a58/manager/0.log" Sep 15 08:05:18.223453 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:18.223426 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-565d545f7b-lpnqs_4e90840b-cab1-407b-8fff-e1feac2c3a58/copy-manifests/0.log" Sep 15 08:05:18.689191 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:18.689166 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt_024f5b7f-74ab-4aaf-9dc9-d9bf9e089707/istio-proxy/0.log" Sep 15 08:05:19.457961 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:19.457880 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b9fc96c88-rrr4r_812aaa8b-f2d2-4363-bf4d-f9ea0254519d/manager/0.log" Sep 15 08:05:19.517784 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:19.517757 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-42jct_6091e72d-1db6-4b32-9420-f1cd119c990b/discovery/0.log" Sep 15 08:05:19.623685 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:19.623647 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-cgt48_11660574-efac-4686-962b-934e3bd76287/payload-processing/0.log" Sep 15 08:05:19.636921 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:19.636890 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c5cb56bb8-jq29f_a951ccbf-4363-44e1-9e9b-45a3ae082e7e/router/0.log" Sep 15 08:05:21.047651 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:21.047611 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c7gqlt_024f5b7f-74ab-4aaf-9dc9-d9bf9e089707/istio-proxy/0.log" Sep 15 08:05:21.878369 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:21.878319 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-42jct_6091e72d-1db6-4b32-9420-f1cd119c990b/discovery/0.log" Sep 15 08:05:21.995948 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:21.995916 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-cgt48_11660574-efac-4686-962b-934e3bd76287/payload-processing/0.log" Sep 15 08:05:22.013593 ip-10-0-143-175 kubenswrapper[2552]: I0915 08:05:22.013559 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c5cb56bb8-jq29f_a951ccbf-4363-44e1-9e9b-45a3ae082e7e/router/0.log"