Sep 06 10:28:08.756135 ip-10-0-139-139 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 06 10:28:08.756146 ip-10-0-139-139 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 06 10:28:08.756156 ip-10-0-139-139 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 06 10:28:08.756369 ip-10-0-139-139 systemd[1]: Failed to start Kubernetes Kubelet. Sep 06 10:28:19.004598 ip-10-0-139-139 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 06 10:28:19.004618 ip-10-0-139-139 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot de79cc22966e4dd5bca9874c20c430ef -- Sep 06 10:30:04.820875 ip-10-0-139-139 systemd[1]: Starting Kubernetes Kubelet... Sep 06 10:30:05.238390 ip-10-0-139-139 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 06 10:30:05.238390 ip-10-0-139-139 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 06 10:30:05.238390 ip-10-0-139-139 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 06 10:30:05.238390 ip-10-0-139-139 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 06 10:30:05.238390 ip-10-0-139-139 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 06 10:30:05.241044 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.240953 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 06 10:30:05.248774 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248755 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 10:30:05.248774 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248773 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248778 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248781 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248785 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248788 2574 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248791 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248794 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248797 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248799 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248802 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248805 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248807 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248810 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248813 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248815 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248818 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248826 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248829 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248831 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248834 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 10:30:05.248844 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248837 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248839 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248842 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248844 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248847 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248850 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248853 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248855 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248858 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248862 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248864 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248867 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248869 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248872 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248878 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248882 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248888 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248891 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248894 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 10:30:05.249355 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248897 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248899 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248903 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248905 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248908 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248910 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248912 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248915 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248919 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248922 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248925 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248928 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248930 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248933 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248936 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248939 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248941 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248944 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248947 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248949 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 10:30:05.249830 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248952 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248955 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248958 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248960 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248963 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248967 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248969 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248971 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248974 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248977 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248980 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248983 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248985 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248988 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248991 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248995 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.248998 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249001 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249003 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249006 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 10:30:05.250309 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249009 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249011 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249014 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249016 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249019 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249021 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249025 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249029 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249032 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249034 2574 feature_gate.go:328] unrecognized feature gate: Example Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249037 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249039 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249042 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249045 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249047 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249050 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249052 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249055 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249057 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 10:30:05.250899 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249829 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249838 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249841 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249844 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249853 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249857 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249859 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249862 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249865 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249868 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249871 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249874 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249877 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249879 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249882 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249884 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249887 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249889 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249894 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 10:30:05.251358 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249897 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249901 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249904 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249907 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249910 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249912 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249915 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249918 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249921 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249924 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249927 2574 feature_gate.go:328] unrecognized feature gate: Example Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249929 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249932 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249935 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249938 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249941 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249943 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249946 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249955 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249957 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 10:30:05.251910 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249966 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249968 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249971 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249974 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249976 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249979 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249981 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249984 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249987 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249989 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249992 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249995 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.249999 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250002 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250004 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250007 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250009 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250012 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250015 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250017 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 10:30:05.252398 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250020 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250022 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250025 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250028 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250030 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250032 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250035 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250037 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250040 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250042 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250045 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250053 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250056 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250060 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250063 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250066 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250068 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250071 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250073 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 10:30:05.252902 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250076 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250079 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250082 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250084 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250087 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250089 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250092 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250095 2574 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250097 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250100 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250102 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250105 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250107 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250110 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250112 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250115 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250118 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250120 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250123 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250125 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 10:30:05.253373 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250128 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250656 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250670 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250674 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250678 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250683 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250687 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250690 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250693 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250697 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250700 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250703 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250706 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250709 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250712 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250715 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250717 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250720 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250723 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 10:30:05.253880 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250725 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250728 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250730 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250733 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250736 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250738 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250741 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250743 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250746 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250749 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250752 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250755 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250757 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250760 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250762 2574 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250767 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250770 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250773 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250776 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250779 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 10:30:05.254341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250781 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250785 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250788 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250791 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250793 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250796 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250798 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250801 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250803 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250806 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250808 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250812 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250816 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250819 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250822 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250825 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250828 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250830 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250833 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 10:30:05.254842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250835 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250838 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250840 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250843 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250845 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250848 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250850 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250853 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250857 2574 feature_gate.go:328] unrecognized feature gate: Example Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250860 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250863 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250871 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250876 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250879 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250881 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250884 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250888 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250891 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250893 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250896 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 10:30:05.255303 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250900 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250905 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250908 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250912 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250915 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250918 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250921 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250924 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250926 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250929 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250932 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250935 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250938 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250941 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250944 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250947 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250949 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250952 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250955 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250957 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 10:30:05.255807 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250963 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.250965 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251050 2574 flags.go:64] FLAG: --address="0.0.0.0" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251086 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251093 2574 flags.go:64] FLAG: --anonymous-auth="true" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251098 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251106 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251109 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251113 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251118 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251121 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251124 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251128 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251131 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251136 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251139 2574 flags.go:64] FLAG: --cgroup-root="" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251142 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251145 2574 flags.go:64] FLAG: --client-ca-file="" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251148 2574 flags.go:64] FLAG: --cloud-provider="external" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251151 2574 flags.go:64] FLAG: --cluster-dns="[]" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251154 2574 flags.go:64] FLAG: --cluster-domain="" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251157 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251160 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251163 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 06 10:30:05.256302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251167 2574 flags.go:64] FLAG: --container-log-max-files="5" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251170 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251173 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251176 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251180 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251183 2574 flags.go:64] FLAG: --contention-profiling="false" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251186 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251189 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251194 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251197 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251202 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251206 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251209 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251212 2574 flags.go:64] FLAG: --enable-load-reader="false" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251215 2574 flags.go:64] FLAG: --enable-server="true" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251218 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251223 2574 flags.go:64] FLAG: --event-burst="100" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251226 2574 flags.go:64] FLAG: --event-qps="50" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251229 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251232 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251234 2574 flags.go:64] FLAG: --eviction-hard="" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251239 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251241 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251244 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251247 2574 flags.go:64] FLAG: --eviction-soft="" Sep 06 10:30:05.256931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251250 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251253 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251256 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251259 2574 flags.go:64] FLAG: --experimental-mounter-path="" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251262 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251265 2574 flags.go:64] FLAG: --fail-swap-on="true" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251268 2574 flags.go:64] FLAG: --feature-gates="" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251272 2574 flags.go:64] FLAG: --file-check-frequency="20s" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251274 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251278 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251281 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251284 2574 flags.go:64] FLAG: --healthz-port="10248" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251287 2574 flags.go:64] FLAG: --help="false" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251290 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251293 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251298 2574 flags.go:64] FLAG: --http-check-frequency="20s" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251301 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251305 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251308 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251312 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251315 2574 flags.go:64] FLAG: --image-service-endpoint="" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251318 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251321 2574 flags.go:64] FLAG: --kube-api-burst="100" Sep 06 10:30:05.257570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251323 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251327 2574 flags.go:64] FLAG: --kube-api-qps="50" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251330 2574 flags.go:64] FLAG: --kube-reserved="" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251332 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251335 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251338 2574 flags.go:64] FLAG: --kubelet-cgroups="" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251341 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251344 2574 flags.go:64] FLAG: --lock-file="" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251347 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251350 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251353 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251359 2574 flags.go:64] FLAG: --log-json-split-stream="false" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251362 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251365 2574 flags.go:64] FLAG: --log-text-split-stream="false" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251368 2574 flags.go:64] FLAG: --logging-format="text" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251371 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251374 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251377 2574 flags.go:64] FLAG: --manifest-url="" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251380 2574 flags.go:64] FLAG: --manifest-url-header="" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251385 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251388 2574 flags.go:64] FLAG: --max-open-files="1000000" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251392 2574 flags.go:64] FLAG: --max-pods="110" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251395 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251398 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251402 2574 flags.go:64] FLAG: --memory-manager-policy="None" Sep 06 10:30:05.258155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251405 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251408 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251411 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251415 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251423 2574 flags.go:64] FLAG: --node-status-max-images="50" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251427 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251430 2574 flags.go:64] FLAG: --oom-score-adj="-999" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251433 2574 flags.go:64] FLAG: --pod-cidr="" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251436 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251442 2574 flags.go:64] FLAG: --pod-manifest-path="" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251444 2574 flags.go:64] FLAG: --pod-max-pids="-1" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251447 2574 flags.go:64] FLAG: --pods-per-core="0" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251450 2574 flags.go:64] FLAG: --port="10250" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251454 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251456 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b18d478e536df128" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251459 2574 flags.go:64] FLAG: --qos-reserved="" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251462 2574 flags.go:64] FLAG: --read-only-port="10255" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251465 2574 flags.go:64] FLAG: --register-node="true" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251468 2574 flags.go:64] FLAG: --register-with-taints="" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251472 2574 flags.go:64] FLAG: --registry-burst="10" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251474 2574 flags.go:64] FLAG: --registry-qps="5" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251477 2574 flags.go:64] FLAG: --reserved-cpus="" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251480 2574 flags.go:64] FLAG: --reserved-memory="" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251484 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251487 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 06 10:30:05.258815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251490 2574 flags.go:64] FLAG: --rotate-certificates="false" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251493 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251495 2574 flags.go:64] FLAG: --runonce="false" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251499 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251502 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251505 2574 flags.go:64] FLAG: --seccomp-default="false" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251508 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251512 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251515 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251518 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251521 2574 flags.go:64] FLAG: --storage-driver-password="root" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251524 2574 flags.go:64] FLAG: --storage-driver-secure="false" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251527 2574 flags.go:64] FLAG: --storage-driver-table="stats" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251530 2574 flags.go:64] FLAG: --storage-driver-user="root" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251533 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251536 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251539 2574 flags.go:64] FLAG: --system-cgroups="" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251542 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251547 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251550 2574 flags.go:64] FLAG: --tls-cert-file="" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251553 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251556 2574 flags.go:64] FLAG: --tls-min-version="" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251559 2574 flags.go:64] FLAG: --tls-private-key-file="" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251562 2574 flags.go:64] FLAG: --topology-manager-policy="none" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251564 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 06 10:30:05.259549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251567 2574 flags.go:64] FLAG: --topology-manager-scope="container" Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251570 2574 flags.go:64] FLAG: --v="2" Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251574 2574 flags.go:64] FLAG: --version="false" Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251591 2574 flags.go:64] FLAG: --vmodule="" Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251595 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.251598 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252099 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252105 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252108 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252111 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252115 2574 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252117 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252120 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252125 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252129 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252131 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252134 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252137 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252139 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252142 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252145 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 10:30:05.260704 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252148 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252150 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252153 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252156 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252158 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252161 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252163 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252166 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252168 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252171 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252173 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252176 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252178 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252181 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252184 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252186 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252189 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252192 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252194 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 10:30:05.261405 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252196 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252199 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252202 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252204 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252207 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252211 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252216 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252219 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252221 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252224 2574 feature_gate.go:328] unrecognized feature gate: Example Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252227 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252229 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252233 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252237 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252240 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252243 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252246 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252250 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252252 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252255 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 10:30:05.261915 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252257 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252260 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252262 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252265 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252268 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252270 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252273 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252275 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252278 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252280 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252283 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252285 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252288 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252290 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252294 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252297 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252299 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252303 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252307 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252310 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 10:30:05.262404 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252312 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252315 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252317 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252320 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252322 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252325 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252328 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252330 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252333 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252336 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252338 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252341 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252343 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252346 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252349 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252352 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252354 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252357 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252359 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252361 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 10:30:05.262913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252364 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252367 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252369 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252372 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.252374 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.253090 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.261985 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.262022 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.262045 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.262050 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262124 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262130 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262133 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262136 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262138 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 10:30:05.263395 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262141 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262144 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262147 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262150 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262152 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262155 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262158 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262161 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262164 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262166 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262169 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262171 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262175 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262177 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262180 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262183 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262185 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262188 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262191 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262193 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 10:30:05.263856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262196 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262198 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262201 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262205 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262208 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262211 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262213 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262216 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262219 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262221 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262224 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262226 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262229 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262232 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262235 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262237 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262240 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262242 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262245 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262248 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 10:30:05.264349 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262251 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262253 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262256 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262260 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262264 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262267 2574 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262270 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262273 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262276 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262279 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262281 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262285 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262288 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262290 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262293 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262297 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262301 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262303 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262306 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 10:30:05.264842 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262309 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262311 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262314 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262316 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262319 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262322 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262324 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262327 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262330 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262333 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262336 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262338 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262342 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262345 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262348 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262352 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262355 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262358 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262361 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 10:30:05.265330 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262363 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262366 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262369 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262372 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262376 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262379 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262382 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262385 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262388 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262391 2574 feature_gate.go:328] unrecognized feature gate: Example Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262394 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262397 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262399 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262402 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262405 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 10:30:05.265803 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262407 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.262413 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262522 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262527 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262530 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262533 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262535 2574 feature_gate.go:328] unrecognized feature gate: Example2 Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262538 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262541 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262544 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262547 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262549 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262552 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262555 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262559 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 06 10:30:05.266168 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262561 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262564 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262568 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262571 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262574 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262595 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262598 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262601 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262604 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262607 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262609 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262612 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262615 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262618 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262621 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262623 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262626 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262629 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262632 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 06 10:30:05.266535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262634 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262637 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262640 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262642 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262645 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262648 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262650 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262653 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262655 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262658 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262662 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262665 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262668 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262672 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262675 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262678 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262680 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262683 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262685 2574 feature_gate.go:328] unrecognized feature gate: Example Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262688 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 06 10:30:05.267018 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262691 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262693 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262696 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262699 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262703 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262706 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262709 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262712 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262715 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262718 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262721 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262724 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262727 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262730 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262733 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262735 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262738 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262741 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262743 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 06 10:30:05.267495 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262746 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262749 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262751 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262754 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262756 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262759 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262761 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262765 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262767 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262770 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262772 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262775 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262777 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262780 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262783 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262785 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262788 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262790 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262793 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262796 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 06 10:30:05.268025 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262798 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 06 10:30:05.268502 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262801 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 06 10:30:05.268502 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262803 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 06 10:30:05.268502 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262806 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 06 10:30:05.268502 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262809 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 06 10:30:05.268502 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262811 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 06 10:30:05.268502 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262814 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 06 10:30:05.268502 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:05.262820 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 06 10:30:05.268502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.262826 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 06 10:30:05.268502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.262963 2574 server.go:962] "Client rotation is on, will bootstrap in background" Sep 06 10:30:05.268502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.267011 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 06 10:30:05.268502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.268100 2574 server.go:1019] "Starting client certificate rotation" Sep 06 10:30:05.268502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.268205 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 06 10:30:05.269066 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.269053 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 06 10:30:05.271150 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.271135 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 06 10:30:05.272278 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.272260 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 06 10:30:05.284189 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.284168 2574 log.go:25] "Validated CRI v1 runtime API" Sep 06 10:30:05.288401 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.288384 2574 log.go:25] "Validated CRI v1 image API" Sep 06 10:30:05.289299 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.289285 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 06 10:30:05.297013 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.296992 2574 fs.go:135] Filesystem UUIDs: map[61c1731a-8269-45d3-b5af-e7f3404f6561:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d44c4fd7-b40b-4084-bec0-f906847e81b4:/dev/nvme0n1p3] Sep 06 10:30:05.297089 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.297011 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 06 10:30:05.297945 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.297929 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 06 10:30:05.303156 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.303033 2574 manager.go:216] Machine: {Timestamp:2026-09-06 10:30:05.302506368 +0000 UTC m=+0.369539439 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099400 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24d478d23537ec4ce743698b8f3fd4 SystemUUID:ec24d478-d235-37ec-4ce7-43698b8f3fd4 BootID:de79cc22-966e-4dd5-bca9-874c20c430ef Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cd:5a:3e:d0:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cd:5a:3e:d0:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:94:02:40:8b:16 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 06 10:30:05.303156 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.303145 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 06 10:30:05.303338 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.303263 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 06 10:30:05.306899 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.306863 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 06 10:30:05.307074 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.306903 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-139.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 06 10:30:05.307762 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.307749 2574 topology_manager.go:138] "Creating topology manager with none policy" Sep 06 10:30:05.307831 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.307767 2574 container_manager_linux.go:309] "Creating device plugin manager" Sep 06 10:30:05.307831 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.307779 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 06 10:30:05.308665 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.308653 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 06 10:30:05.310308 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.310295 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 06 10:30:05.315187 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.315171 2574 state_mem.go:36] "Initialized new in-memory state store" Sep 06 10:30:05.315335 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.315324 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 06 10:30:05.315390 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.315364 2574 kubelet.go:486] "Attempting to sync node with API server" Sep 06 10:30:05.315390 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.315381 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 06 10:30:05.315489 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.315405 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 06 10:30:05.315489 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.315421 2574 kubelet.go:398] "Adding apiserver pod source" Sep 06 10:30:05.315489 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.315433 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 06 10:30:05.315884 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.315853 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 06 10:30:05.315884 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.315876 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 06 10:30:05.319078 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.319058 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 06 10:30:05.321489 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.321471 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gnn4f" Sep 06 10:30:05.322404 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.322392 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 06 10:30:05.322452 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.322421 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 06 10:30:05.322613 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.322603 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 06 10:30:05.322661 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.322619 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 06 10:30:05.322661 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.322625 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 06 10:30:05.322661 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.322630 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 06 10:30:05.322661 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.322639 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 06 10:30:05.322661 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.322647 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 06 10:30:05.323155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.323145 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 06 10:30:05.323155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.323154 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 06 10:30:05.323233 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.323161 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 06 10:30:05.323233 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.323167 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 06 10:30:05.323233 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.323184 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 06 10:30:05.323233 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.323192 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 06 10:30:05.324765 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.324752 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 06 10:30:05.324765 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.324763 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 06 10:30:05.327634 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.327614 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-139.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 06 10:30:05.328340 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.328317 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-139.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 06 10:30:05.328401 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.328320 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 06 10:30:05.328810 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.328793 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gnn4f" Sep 06 10:30:05.328855 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.328828 2574 server.go:1268] "Started kubelet" Sep 06 10:30:05.328990 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.328935 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 06 10:30:05.329062 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.329009 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 06 10:30:05.329151 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.329086 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 06 10:30:05.330944 ip-10-0-139-139 systemd[1]: Started Kubernetes Kubelet. Sep 06 10:30:05.331874 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.331447 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 06 10:30:05.332261 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.332248 2574 server.go:310] "Adding debug handlers to kubelet server" Sep 06 10:30:05.334456 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.334438 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 06 10:30:05.336771 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.335746 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-139.ec2.internal.18d2b4a0efb3940e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-139.ec2.internal,UID:ip-10-0-139-139.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-139.ec2.internal,},FirstTimestamp:2026-09-06 10:30:05.328798734 +0000 UTC m=+0.395831808,LastTimestamp:2026-09-06 10:30:05.328798734 +0000 UTC m=+0.395831808,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-139.ec2.internal,}" Sep 06 10:30:05.337105 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.336907 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 06 10:30:05.337105 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.336923 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 06 10:30:05.337540 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.337519 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-139.ec2.internal\" not found" Sep 06 10:30:05.337880 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.337856 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 06 10:30:05.337880 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.337870 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 06 10:30:05.338060 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.337896 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 06 10:30:05.338060 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.338005 2574 factory.go:55] Registering systemd factory Sep 06 10:30:05.338060 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.338021 2574 reconstruct.go:97] "Volume reconstruction finished" Sep 06 10:30:05.338060 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.338030 2574 reconciler.go:29] "Reconciler: start to sync state" Sep 06 10:30:05.338060 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.338022 2574 factory.go:223] Registration of the systemd container factory successfully Sep 06 10:30:05.338305 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.338288 2574 factory.go:153] Registering CRI-O factory Sep 06 10:30:05.338305 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.338305 2574 factory.go:223] Registration of the crio container factory successfully Sep 06 10:30:05.338443 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.338367 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 06 10:30:05.338443 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.338386 2574 factory.go:103] Registering Raw factory Sep 06 10:30:05.338443 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.338404 2574 manager.go:1196] Started watching for new ooms in manager Sep 06 10:30:05.338920 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.338865 2574 manager.go:349] Starting recovery of all containers Sep 06 10:30:05.341454 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.341429 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 10:30:05.343925 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.343894 2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-139.ec2.internal\" not found" node="ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.347137 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.347110 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 06 10:30:05.350947 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.350923 2574 manager.go:354] Recovery completed Sep 06 10:30:05.355980 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.355966 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 10:30:05.357984 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.357968 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientMemory" Sep 06 10:30:05.358045 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.357998 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 10:30:05.358045 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.358009 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientPID" Sep 06 10:30:05.360334 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.360318 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 06 10:30:05.360334 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.360331 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 06 10:30:05.360450 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.360339 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 10:30:05.360450 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.360349 2574 state_mem.go:36] "Initialized new in-memory state store" Sep 06 10:30:05.361227 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.361210 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientMemory" Sep 06 10:30:05.361307 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.361244 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 10:30:05.361307 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.361258 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientPID" Sep 06 10:30:05.363441 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.363428 2574 policy_none.go:49] "None policy: Start" Sep 06 10:30:05.363485 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.363447 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 06 10:30:05.363485 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.363460 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 06 10:30:05.365798 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.365782 2574 policy_none.go:47] "Start" Sep 06 10:30:05.393180 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.393163 2574 manager.go:337] "Starting Device Plugin manager" Sep 06 10:30:05.393286 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.393199 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 06 10:30:05.393286 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.393209 2574 server.go:87] "Starting device plugin registration server" Sep 06 10:30:05.393564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.393548 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 06 10:30:05.393829 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.393565 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 06 10:30:05.421563 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.393895 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 06 10:30:05.421563 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.393973 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 06 10:30:05.421563 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.393981 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 06 10:30:05.421563 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.394505 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 06 10:30:05.421563 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.394570 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-139.ec2.internal\" not found" Sep 06 10:30:05.435278 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.435247 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 06 10:30:05.435380 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.435308 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 06 10:30:05.435380 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.435344 2574 kubelet.go:2448] "Starting kubelet main sync loop" Sep 06 10:30:05.435458 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.435406 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 06 10:30:05.437859 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.437834 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 10:30:05.494728 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.494620 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 10:30:05.495773 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.495750 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientMemory" Sep 06 10:30:05.495885 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.495790 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 10:30:05.495885 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.495805 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientPID" Sep 06 10:30:05.495885 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.495835 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.504032 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.504011 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.504131 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.504042 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-139.ec2.internal\": node \"ip-10-0-139-139.ec2.internal\" not found" Sep 06 10:30:05.513788 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.513767 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-139.ec2.internal\" not found" Sep 06 10:30:05.536460 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.536436 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-139.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal"] Sep 06 10:30:05.536547 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.536528 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 10:30:05.538054 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.538021 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientMemory" Sep 06 10:30:05.538147 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.538060 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 10:30:05.538147 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.538087 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientPID" Sep 06 10:30:05.538220 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.538203 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d442c7b86c5fc53cff3e2ee8151b31bc-config\") pod \"kube-apiserver-proxy-ip-10-0-139-139.ec2.internal\" (UID: \"d442c7b86c5fc53cff3e2ee8151b31bc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.540394 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.540380 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 10:30:05.540597 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.540560 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.540635 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.540617 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 10:30:05.541299 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.541281 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientMemory" Sep 06 10:30:05.541299 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.541292 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientMemory" Sep 06 10:30:05.541428 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.541312 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 10:30:05.541428 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.541324 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 10:30:05.541428 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.541336 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientPID" Sep 06 10:30:05.541428 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.541325 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientPID" Sep 06 10:30:05.543666 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.543650 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.543727 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.543685 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 06 10:30:05.544532 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.544518 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientMemory" Sep 06 10:30:05.544624 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.544538 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasNoDiskPressure" Sep 06 10:30:05.544624 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.544551 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeHasSufficientPID" Sep 06 10:30:05.560202 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.560181 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-139.ec2.internal\" not found" node="ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.564390 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.564375 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-139.ec2.internal\" not found" node="ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.614426 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.614399 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-139.ec2.internal\" not found" Sep 06 10:30:05.638419 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.638397 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d442c7b86c5fc53cff3e2ee8151b31bc-config\") pod \"kube-apiserver-proxy-ip-10-0-139-139.ec2.internal\" (UID: \"d442c7b86c5fc53cff3e2ee8151b31bc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.638477 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.638435 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d442c7b86c5fc53cff3e2ee8151b31bc-config\") pod \"kube-apiserver-proxy-ip-10-0-139-139.ec2.internal\" (UID: \"d442c7b86c5fc53cff3e2ee8151b31bc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.715045 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.715001 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-139.ec2.internal\" not found" Sep 06 10:30:05.739296 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.739268 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03d807d2b2681f19d290cbd8bf76f7e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal\" (UID: \"03d807d2b2681f19d290cbd8bf76f7e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.739352 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.739302 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03d807d2b2681f19d290cbd8bf76f7e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal\" (UID: \"03d807d2b2681f19d290cbd8bf76f7e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.815670 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.815570 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-139.ec2.internal\" not found" Sep 06 10:30:05.840037 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.840008 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03d807d2b2681f19d290cbd8bf76f7e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal\" (UID: \"03d807d2b2681f19d290cbd8bf76f7e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.840106 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.840042 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03d807d2b2681f19d290cbd8bf76f7e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal\" (UID: \"03d807d2b2681f19d290cbd8bf76f7e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.840106 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.840069 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03d807d2b2681f19d290cbd8bf76f7e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal\" (UID: \"03d807d2b2681f19d290cbd8bf76f7e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.840106 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.840085 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/03d807d2b2681f19d290cbd8bf76f7e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal\" (UID: \"03d807d2b2681f19d290cbd8bf76f7e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.863183 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.863159 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.866993 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:05.866970 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" Sep 06 10:30:05.916649 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:05.916621 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-139.ec2.internal\" not found" Sep 06 10:30:07.501724 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:07.501690 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-139.ec2.internal\" not found" Sep 06 10:30:07.602248 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:07.602168 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-139.ec2.internal\" not found" Sep 06 10:30:07.702959 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:07.702927 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-139.ec2.internal\" not found" Sep 06 10:30:07.752413 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.752384 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 06 10:30:07.752901 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.752545 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 06 10:30:07.752901 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.752591 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 06 10:30:07.803844 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:07.803800 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-139.ec2.internal\" not found" Sep 06 10:30:07.819728 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.819693 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-05 10:25:06 +0000 UTC" deadline="2028-03-11 21:47:11.217645111 +0000 UTC" Sep 06 10:30:07.819728 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.819722 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13259h17m3.397926784s" Sep 06 10:30:07.848171 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.847487 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 10:30:07.853674 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:07.853608 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd442c7b86c5fc53cff3e2ee8151b31bc.slice/crio-2dee728a1c201cb390a97868a0a6ff60a87918b622306e59fb848f8b1c627611 WatchSource:0}: Error finding container 2dee728a1c201cb390a97868a0a6ff60a87918b622306e59fb848f8b1c627611: Status 404 returned error can't find the container with id 2dee728a1c201cb390a97868a0a6ff60a87918b622306e59fb848f8b1c627611 Sep 06 10:30:07.854059 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:07.854038 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d807d2b2681f19d290cbd8bf76f7e7.slice/crio-8057fb0e787ad03dfab8ee538a54a81498f729ff7405128a07962a12215d2d23 WatchSource:0}: Error finding container 8057fb0e787ad03dfab8ee538a54a81498f729ff7405128a07962a12215d2d23: Status 404 returned error can't find the container with id 8057fb0e787ad03dfab8ee538a54a81498f729ff7405128a07962a12215d2d23 Sep 06 10:30:07.863599 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.863582 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 10:30:07.882920 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.882900 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 10:30:07.922341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.922110 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-139.ec2.internal" Sep 06 10:30:07.924018 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.923977 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-139.ec2.internal" event={"ID":"d442c7b86c5fc53cff3e2ee8151b31bc","Type":"ContainerStarted","Data":"2dee728a1c201cb390a97868a0a6ff60a87918b622306e59fb848f8b1c627611"} Sep 06 10:30:07.924987 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.924963 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" event={"ID":"03d807d2b2681f19d290cbd8bf76f7e7","Type":"ContainerStarted","Data":"8057fb0e787ad03dfab8ee538a54a81498f729ff7405128a07962a12215d2d23"} Sep 06 10:30:07.932177 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.932159 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 06 10:30:07.933283 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.933268 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" Sep 06 10:30:07.942991 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:07.942975 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 06 10:30:08.801315 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.801276 2574 apiserver.go:52] "Watching apiserver" Sep 06 10:30:08.812608 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.812570 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 06 10:30:08.813699 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.813668 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-hn5z4","openshift-image-registry/node-ca-4h9s4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal","openshift-multus/multus-additional-cni-plugins-hfgzs","openshift-network-operator/iptables-alerter-gfwjq","kube-system/global-pull-secret-syncer-n424n","kube-system/kube-apiserver-proxy-ip-10-0-139-139.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx","openshift-cluster-node-tuning-operator/tuned-2hg8p","openshift-multus/multus-6rcdb","openshift-multus/network-metrics-daemon-tg7pf","openshift-network-diagnostics/network-check-target-lrpg9","openshift-ovn-kubernetes/ovnkube-node-cj6jp","kube-system/konnectivity-agent-ggl9h"] Sep 06 10:30:08.816600 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.816518 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gfwjq" Sep 06 10:30:08.818981 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.818953 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.819081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.819019 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 06 10:30:08.819340 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.818962 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 06 10:30:08.819576 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.819560 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 06 10:30:08.819657 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.819591 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rc7zw\"" Sep 06 10:30:08.820802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.820727 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 06 10:30:08.820902 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.820880 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 06 10:30:08.821251 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.821231 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.822007 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.821788 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 06 10:30:08.822007 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.821932 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bwn8j\"" Sep 06 10:30:08.822435 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.822416 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 06 10:30:08.823260 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.823241 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 06 10:30:08.823812 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.823792 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:08.823892 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:08.823864 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:08.823951 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.823904 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 06 10:30:08.824195 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.824176 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 06 10:30:08.824571 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.824409 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8r82j\"" Sep 06 10:30:08.824571 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.824461 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 06 10:30:08.824719 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.824658 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 06 10:30:08.826076 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.826055 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hn5z4" Sep 06 10:30:08.828560 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.828539 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 06 10:30:08.830263 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.828684 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g722g\"" Sep 06 10:30:08.830263 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.828065 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 06 10:30:08.830263 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.829102 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4h9s4" Sep 06 10:30:08.831577 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.831556 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v2mjb\"" Sep 06 10:30:08.833074 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833054 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.833184 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833137 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 06 10:30:08.833372 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833344 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 06 10:30:08.833527 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833054 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 06 10:30:08.833694 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833676 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcq5t\" (UniqueName: \"kubernetes.io/projected/5490127e-cffd-4c95-bccc-8709ee887aaf-kube-api-access-mcq5t\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.833766 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833706 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d037fa14-5fbf-4635-92df-3b0117254fca-dbus\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:08.833825 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833782 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85ed5170-128c-493b-8737-94af607505c6-host-slash\") pod \"iptables-alerter-gfwjq\" (UID: \"85ed5170-128c-493b-8737-94af607505c6\") " pod="openshift-network-operator/iptables-alerter-gfwjq" Sep 06 10:30:08.833878 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833823 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-device-dir\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.833878 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833851 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52513a46-7d58-48a5-a6a9-7633b34482c7-system-cni-dir\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.833979 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833878 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52513a46-7d58-48a5-a6a9-7633b34482c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.833979 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833905 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpdlq\" (UniqueName: \"kubernetes.io/projected/52513a46-7d58-48a5-a6a9-7633b34482c7-kube-api-access-wpdlq\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.833979 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833929 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d037fa14-5fbf-4635-92df-3b0117254fca-kubelet-config\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:08.833979 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833973 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.834194 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.833997 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-socket-dir\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.834194 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.834045 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-registration-dir\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.834194 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.834070 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-etc-selinux\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.834194 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.834093 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52513a46-7d58-48a5-a6a9-7633b34482c7-cnibin\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.834194 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.834118 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/85ed5170-128c-493b-8737-94af607505c6-iptables-alerter-script\") pod \"iptables-alerter-gfwjq\" (UID: \"85ed5170-128c-493b-8737-94af607505c6\") " pod="openshift-network-operator/iptables-alerter-gfwjq" Sep 06 10:30:08.834194 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.834160 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvjhn\" (UniqueName: \"kubernetes.io/projected/85ed5170-128c-493b-8737-94af607505c6-kube-api-access-wvjhn\") pod \"iptables-alerter-gfwjq\" (UID: \"85ed5170-128c-493b-8737-94af607505c6\") " pod="openshift-network-operator/iptables-alerter-gfwjq" Sep 06 10:30:08.834462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.834219 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-sys-fs\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.834462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.834295 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52513a46-7d58-48a5-a6a9-7633b34482c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.834462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.834337 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/52513a46-7d58-48a5-a6a9-7633b34482c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.834462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.834360 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/52513a46-7d58-48a5-a6a9-7633b34482c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.834462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.834387 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:08.835716 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.835465 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 06 10:30:08.835716 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.835489 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 06 10:30:08.835716 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.835533 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zhgbb\"" Sep 06 10:30:08.835716 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.835592 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.837011 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.836987 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 06 10:30:08.837490 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.837472 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 06 10:30:08.838067 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.838047 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bmjwh\"" Sep 06 10:30:08.838326 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.838299 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:08.838477 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:08.838376 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:08.840381 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.840360 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:08.840468 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:08.840426 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:08.842849 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.842829 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.845315 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.845231 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ggl9h" Sep 06 10:30:08.845414 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.845337 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 06 10:30:08.845414 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.845348 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 06 10:30:08.845414 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.845364 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 06 10:30:08.845561 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.845341 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 06 10:30:08.845853 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.845835 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 06 10:30:08.845924 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.845905 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5gptd\"" Sep 06 10:30:08.845974 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.845934 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 06 10:30:08.847246 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.847228 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kf56d\"" Sep 06 10:30:08.847576 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.847540 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 06 10:30:08.847666 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.847615 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 06 10:30:08.854239 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.854216 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wqlps" Sep 06 10:30:08.861688 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.861661 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wqlps" Sep 06 10:30:08.924879 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.924845 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 06 10:30:08.935610 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935579 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-sys-fs\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.935610 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935613 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52513a46-7d58-48a5-a6a9-7633b34482c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.935854 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935641 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-multus-conf-dir\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.935854 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935665 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g5bd\" (UniqueName: \"kubernetes.io/projected/a28d14f7-334e-4b14-b75b-14619d4c4f4f-kube-api-access-8g5bd\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.935854 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935689 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-modprobe-d\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.935854 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935712 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d3f60912-3413-4b9f-acb3-8d38dd276a82-konnectivity-ca\") pod \"konnectivity-agent-ggl9h\" (UID: \"d3f60912-3413-4b9f-acb3-8d38dd276a82\") " pod="kube-system/konnectivity-agent-ggl9h" Sep 06 10:30:08.935854 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935715 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-sys-fs\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.935854 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935734 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-cni-bin\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.935854 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935757 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2da0a8a3-afbc-41fc-bd56-69500eb10934-hosts-file\") pod \"node-resolver-hn5z4\" (UID: \"2da0a8a3-afbc-41fc-bd56-69500eb10934\") " pod="openshift-dns/node-resolver-hn5z4" Sep 06 10:30:08.935854 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935780 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41f8d38b-5235-4d1e-9840-2931fc5cd2b6-serviceca\") pod \"node-ca-4h9s4\" (UID: \"41f8d38b-5235-4d1e-9840-2931fc5cd2b6\") " pod="openshift-image-registry/node-ca-4h9s4" Sep 06 10:30:08.935854 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935803 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-var-lib-kubelet\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.935854 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935827 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87de50d6-3044-4202-bbee-c6e185f89313-etc-tuned\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935868 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-device-dir\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935891 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpfh8\" (UniqueName: \"kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8\") pod \"network-check-target-lrpg9\" (UID: \"dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4\") " pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935927 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-cni-netd\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935953 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-socket-dir\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935968 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-device-dir\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.935990 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-run-k8s-cni-cncf-io\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936037 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-var-lib-openvswitch\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936065 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936082 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41f8d38b-5235-4d1e-9840-2931fc5cd2b6-host\") pod \"node-ca-4h9s4\" (UID: \"41f8d38b-5235-4d1e-9840-2931fc5cd2b6\") " pod="openshift-image-registry/node-ca-4h9s4" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936111 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/52513a46-7d58-48a5-a6a9-7633b34482c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936144 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-socket-dir\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936154 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/52513a46-7d58-48a5-a6a9-7633b34482c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936181 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a28d14f7-334e-4b14-b75b-14619d4c4f4f-multus-daemon-config\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936204 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stgbv\" (UniqueName: \"kubernetes.io/projected/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-kube-api-access-stgbv\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936226 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-kubelet\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.936313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936248 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87de50d6-3044-4202-bbee-c6e185f89313-tmp\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936273 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcq5t\" (UniqueName: \"kubernetes.io/projected/5490127e-cffd-4c95-bccc-8709ee887aaf-kube-api-access-mcq5t\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936297 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-systemd-units\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936320 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-run-openvswitch\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936319 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52513a46-7d58-48a5-a6a9-7633b34482c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936343 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lcxw\" (UniqueName: \"kubernetes.io/projected/41f8d38b-5235-4d1e-9840-2931fc5cd2b6-kube-api-access-4lcxw\") pod \"node-ca-4h9s4\" (UID: \"41f8d38b-5235-4d1e-9840-2931fc5cd2b6\") " pod="openshift-image-registry/node-ca-4h9s4" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936404 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85ed5170-128c-493b-8737-94af607505c6-host-slash\") pod \"iptables-alerter-gfwjq\" (UID: \"85ed5170-128c-493b-8737-94af607505c6\") " pod="openshift-network-operator/iptables-alerter-gfwjq" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936442 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52513a46-7d58-48a5-a6a9-7633b34482c7-system-cni-dir\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936467 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a28d14f7-334e-4b14-b75b-14619d4c4f4f-cni-binary-copy\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936504 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85ed5170-128c-493b-8737-94af607505c6-host-slash\") pod \"iptables-alerter-gfwjq\" (UID: \"85ed5170-128c-493b-8737-94af607505c6\") " pod="openshift-network-operator/iptables-alerter-gfwjq" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936490 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-var-lib-kubelet\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936508 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52513a46-7d58-48a5-a6a9-7633b34482c7-system-cni-dir\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936555 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/710a94f2-acb3-4a5e-98c8-e8aab73e1547-env-overrides\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936589 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-registration-dir\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936619 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/52513a46-7d58-48a5-a6a9-7633b34482c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936616 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-etc-selinux\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.937026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936680 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-registration-dir\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936691 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52513a46-7d58-48a5-a6a9-7633b34482c7-cnibin\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936717 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-node-log\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936744 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-run-ovn-kubernetes\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936743 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52513a46-7d58-48a5-a6a9-7633b34482c7-cnibin\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936725 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/52513a46-7d58-48a5-a6a9-7633b34482c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936780 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-etc-selinux\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936773 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/710a94f2-acb3-4a5e-98c8-e8aab73e1547-ovnkube-script-lib\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936820 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-run\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936838 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-sys\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936866 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:08.936949 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.936970 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-systemd\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:08.937020 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret podName:d037fa14-5fbf-4635-92df-3b0117254fca nodeName:}" failed. No retries permitted until 2026-09-06 10:30:09.437003548 +0000 UTC m=+3.019493170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret") pod "global-pull-secret-syncer-n424n" (UID: "d037fa14-5fbf-4635-92df-3b0117254fca") : object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937099 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d3f60912-3413-4b9f-acb3-8d38dd276a82-agent-certs\") pod \"konnectivity-agent-ggl9h\" (UID: \"d3f60912-3413-4b9f-acb3-8d38dd276a82\") " pod="kube-system/konnectivity-agent-ggl9h" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937153 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d037fa14-5fbf-4635-92df-3b0117254fca-dbus\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:08.937700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937185 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-run-netns\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937199 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/710a94f2-acb3-4a5e-98c8-e8aab73e1547-ovn-node-metrics-cert\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937215 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-sysconfig\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937237 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-host\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937263 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52513a46-7d58-48a5-a6a9-7633b34482c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937283 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d037fa14-5fbf-4635-92df-3b0117254fca-kubelet-config\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937297 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-log-socket\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937310 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/710a94f2-acb3-4a5e-98c8-e8aab73e1547-ovnkube-config\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937337 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d037fa14-5fbf-4635-92df-3b0117254fca-dbus\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937345 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-sysctl-conf\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937349 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d037fa14-5fbf-4635-92df-3b0117254fca-kubelet-config\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937371 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-system-cni-dir\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937386 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-multus-socket-dir-parent\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937400 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-run-multus-certs\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937418 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52513a46-7d58-48a5-a6a9-7633b34482c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937448 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-run-systemd\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937475 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/85ed5170-128c-493b-8737-94af607505c6-iptables-alerter-script\") pod \"iptables-alerter-gfwjq\" (UID: \"85ed5170-128c-493b-8737-94af607505c6\") " pod="openshift-network-operator/iptables-alerter-gfwjq" Sep 06 10:30:08.938386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937497 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvjhn\" (UniqueName: \"kubernetes.io/projected/85ed5170-128c-493b-8737-94af607505c6-kube-api-access-wvjhn\") pod \"iptables-alerter-gfwjq\" (UID: \"85ed5170-128c-493b-8737-94af607505c6\") " pod="openshift-network-operator/iptables-alerter-gfwjq" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937588 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-var-lib-cni-bin\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937695 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-var-lib-cni-multus\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937727 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-etc-kubernetes\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937751 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-etc-openvswitch\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937786 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-run-ovn\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937810 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2da0a8a3-afbc-41fc-bd56-69500eb10934-tmp-dir\") pod \"node-resolver-hn5z4\" (UID: \"2da0a8a3-afbc-41fc-bd56-69500eb10934\") " pod="openshift-dns/node-resolver-hn5z4" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937831 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-cnibin\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937877 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-hostroot\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937901 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-slash\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937924 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-run-netns\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937948 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q42k5\" (UniqueName: \"kubernetes.io/projected/87de50d6-3044-4202-bbee-c6e185f89313-kube-api-access-q42k5\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937978 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpdlq\" (UniqueName: \"kubernetes.io/projected/52513a46-7d58-48a5-a6a9-7633b34482c7-kube-api-access-wpdlq\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.937992 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/85ed5170-128c-493b-8737-94af607505c6-iptables-alerter-script\") pod \"iptables-alerter-gfwjq\" (UID: \"85ed5170-128c-493b-8737-94af607505c6\") " pod="openshift-network-operator/iptables-alerter-gfwjq" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.938002 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-multus-cni-dir\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.938045 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhctk\" (UniqueName: \"kubernetes.io/projected/2da0a8a3-afbc-41fc-bd56-69500eb10934-kube-api-access-fhctk\") pod \"node-resolver-hn5z4\" (UID: \"2da0a8a3-afbc-41fc-bd56-69500eb10934\") " pod="openshift-dns/node-resolver-hn5z4" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.938072 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-sysctl-d\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.939055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.938093 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-lib-modules\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.939721 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.938142 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.939721 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.938166 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:08.939721 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.938189 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp76q\" (UniqueName: \"kubernetes.io/projected/710a94f2-acb3-4a5e-98c8-e8aab73e1547-kube-api-access-gp76q\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:08.939721 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.938211 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-kubernetes\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:08.939721 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.938250 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5490127e-cffd-4c95-bccc-8709ee887aaf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.943491 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.943469 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 06 10:30:08.947630 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.947601 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcq5t\" (UniqueName: \"kubernetes.io/projected/5490127e-cffd-4c95-bccc-8709ee887aaf-kube-api-access-mcq5t\") pod \"aws-ebs-csi-driver-node-vrlbx\" (UID: \"5490127e-cffd-4c95-bccc-8709ee887aaf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:08.952214 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.952183 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvjhn\" (UniqueName: \"kubernetes.io/projected/85ed5170-128c-493b-8737-94af607505c6-kube-api-access-wvjhn\") pod \"iptables-alerter-gfwjq\" (UID: \"85ed5170-128c-493b-8737-94af607505c6\") " pod="openshift-network-operator/iptables-alerter-gfwjq" Sep 06 10:30:08.952347 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:08.952215 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpdlq\" (UniqueName: \"kubernetes.io/projected/52513a46-7d58-48a5-a6a9-7633b34482c7-kube-api-access-wpdlq\") pod \"multus-additional-cni-plugins-hfgzs\" (UID: \"52513a46-7d58-48a5-a6a9-7633b34482c7\") " pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:09.038979 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.038941 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-run-openvswitch\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.039181 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.038988 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lcxw\" (UniqueName: \"kubernetes.io/projected/41f8d38b-5235-4d1e-9840-2931fc5cd2b6-kube-api-access-4lcxw\") pod \"node-ca-4h9s4\" (UID: \"41f8d38b-5235-4d1e-9840-2931fc5cd2b6\") " pod="openshift-image-registry/node-ca-4h9s4" Sep 06 10:30:09.039181 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039016 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a28d14f7-334e-4b14-b75b-14619d4c4f4f-cni-binary-copy\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.039181 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039038 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-var-lib-kubelet\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.039181 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039066 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-run-openvswitch\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.039181 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039092 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-var-lib-kubelet\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.039181 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/710a94f2-acb3-4a5e-98c8-e8aab73e1547-env-overrides\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.039459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039209 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-node-log\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.039459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039227 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-run-ovn-kubernetes\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.039459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039245 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/710a94f2-acb3-4a5e-98c8-e8aab73e1547-ovnkube-script-lib\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.039459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039264 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-run\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.039459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039286 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-sys\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.039459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039291 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-node-log\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.039459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039328 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-run-ovn-kubernetes\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.039459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039345 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-sys\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.039459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-systemd\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.039459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039384 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d3f60912-3413-4b9f-acb3-8d38dd276a82-agent-certs\") pod \"konnectivity-agent-ggl9h\" (UID: \"d3f60912-3413-4b9f-acb3-8d38dd276a82\") " pod="kube-system/konnectivity-agent-ggl9h" Sep 06 10:30:09.039459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039345 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-run\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.039459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039419 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-run-netns\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039466 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-run-netns\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039476 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-systemd\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039571 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/710a94f2-acb3-4a5e-98c8-e8aab73e1547-ovn-node-metrics-cert\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039601 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-sysconfig\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039645 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-host\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039665 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-log-socket\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039680 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/710a94f2-acb3-4a5e-98c8-e8aab73e1547-ovnkube-config\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039686 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-sysconfig\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039696 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-sysctl-conf\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039730 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-system-cni-dir\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039739 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-log-socket\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039757 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-multus-socket-dir-parent\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039781 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-run-multus-certs\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039792 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-host\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039807 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-run-systemd\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039829 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-sysctl-conf\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039833 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/710a94f2-acb3-4a5e-98c8-e8aab73e1547-env-overrides\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.040019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039834 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-var-lib-cni-bin\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039869 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-var-lib-cni-bin\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039871 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-var-lib-cni-multus\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039894 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-multus-socket-dir-parent\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039898 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-var-lib-cni-multus\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039907 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-etc-kubernetes\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039930 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-run-multus-certs\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039934 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-etc-kubernetes\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039953 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/710a94f2-acb3-4a5e-98c8-e8aab73e1547-ovnkube-script-lib\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.039970 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-run-systemd\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040013 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-etc-openvswitch\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040039 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-run-ovn\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040064 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2da0a8a3-afbc-41fc-bd56-69500eb10934-tmp-dir\") pod \"node-resolver-hn5z4\" (UID: \"2da0a8a3-afbc-41fc-bd56-69500eb10934\") " pod="openshift-dns/node-resolver-hn5z4" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040085 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-system-cni-dir\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040087 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-cnibin\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040145 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-hostroot\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040170 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-slash\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040175 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-etc-openvswitch\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.040875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040145 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-cnibin\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040222 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-hostroot\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040233 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-run-netns\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040257 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-run-ovn\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040270 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q42k5\" (UniqueName: \"kubernetes.io/projected/87de50d6-3044-4202-bbee-c6e185f89313-kube-api-access-q42k5\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040283 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a28d14f7-334e-4b14-b75b-14619d4c4f4f-cni-binary-copy\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040304 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-run-netns\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040326 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/710a94f2-acb3-4a5e-98c8-e8aab73e1547-ovnkube-config\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040332 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-slash\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040406 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-multus-cni-dir\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040431 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2da0a8a3-afbc-41fc-bd56-69500eb10934-tmp-dir\") pod \"node-resolver-hn5z4\" (UID: \"2da0a8a3-afbc-41fc-bd56-69500eb10934\") " pod="openshift-dns/node-resolver-hn5z4" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040433 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhctk\" (UniqueName: \"kubernetes.io/projected/2da0a8a3-afbc-41fc-bd56-69500eb10934-kube-api-access-fhctk\") pod \"node-resolver-hn5z4\" (UID: \"2da0a8a3-afbc-41fc-bd56-69500eb10934\") " pod="openshift-dns/node-resolver-hn5z4" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040482 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-sysctl-d\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040518 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-lib-modules\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040543 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040544 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-multus-cni-dir\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040567 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp76q\" (UniqueName: \"kubernetes.io/projected/710a94f2-acb3-4a5e-98c8-e8aab73e1547-kube-api-access-gp76q\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040592 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-kubernetes\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.041780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040614 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-multus-conf-dir\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040642 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g5bd\" (UniqueName: \"kubernetes.io/projected/a28d14f7-334e-4b14-b75b-14619d4c4f4f-kube-api-access-8g5bd\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040644 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-lib-modules\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.040661 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040687 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-multus-conf-dir\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040691 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-kubernetes\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040613 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-sysctl-d\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040664 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-modprobe-d\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040729 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-etc-modprobe-d\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.040737 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs podName:23d754da-9be5-4fe4-9d31-0ce5f1c3b63e nodeName:}" failed. No retries permitted until 2026-09-06 10:30:09.540716421 +0000 UTC m=+3.123206032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs") pod "network-metrics-daemon-tg7pf" (UID: "23d754da-9be5-4fe4-9d31-0ce5f1c3b63e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040775 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d3f60912-3413-4b9f-acb3-8d38dd276a82-konnectivity-ca\") pod \"konnectivity-agent-ggl9h\" (UID: \"d3f60912-3413-4b9f-acb3-8d38dd276a82\") " pod="kube-system/konnectivity-agent-ggl9h" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040799 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-cni-bin\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040825 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2da0a8a3-afbc-41fc-bd56-69500eb10934-hosts-file\") pod \"node-resolver-hn5z4\" (UID: \"2da0a8a3-afbc-41fc-bd56-69500eb10934\") " pod="openshift-dns/node-resolver-hn5z4" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040850 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41f8d38b-5235-4d1e-9840-2931fc5cd2b6-serviceca\") pod \"node-ca-4h9s4\" (UID: \"41f8d38b-5235-4d1e-9840-2931fc5cd2b6\") " pod="openshift-image-registry/node-ca-4h9s4" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040857 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-cni-bin\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040873 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-var-lib-kubelet\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040910 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2da0a8a3-afbc-41fc-bd56-69500eb10934-hosts-file\") pod \"node-resolver-hn5z4\" (UID: \"2da0a8a3-afbc-41fc-bd56-69500eb10934\") " pod="openshift-dns/node-resolver-hn5z4" Sep 06 10:30:09.042512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040919 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87de50d6-3044-4202-bbee-c6e185f89313-etc-tuned\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040946 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpfh8\" (UniqueName: \"kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8\") pod \"network-check-target-lrpg9\" (UID: \"dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4\") " pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040970 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-cni-netd\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040980 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87de50d6-3044-4202-bbee-c6e185f89313-var-lib-kubelet\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.040996 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-run-k8s-cni-cncf-io\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041019 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-var-lib-openvswitch\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041082 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041108 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41f8d38b-5235-4d1e-9840-2931fc5cd2b6-host\") pod \"node-ca-4h9s4\" (UID: \"41f8d38b-5235-4d1e-9840-2931fc5cd2b6\") " pod="openshift-image-registry/node-ca-4h9s4" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041151 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a28d14f7-334e-4b14-b75b-14619d4c4f4f-multus-daemon-config\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041177 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stgbv\" (UniqueName: \"kubernetes.io/projected/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-kube-api-access-stgbv\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041201 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-kubelet\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041258 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-kubelet\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041284 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041309 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41f8d38b-5235-4d1e-9840-2931fc5cd2b6-host\") pod \"node-ca-4h9s4\" (UID: \"41f8d38b-5235-4d1e-9840-2931fc5cd2b6\") " pod="openshift-image-registry/node-ca-4h9s4" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041312 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41f8d38b-5235-4d1e-9840-2931fc5cd2b6-serviceca\") pod \"node-ca-4h9s4\" (UID: \"41f8d38b-5235-4d1e-9840-2931fc5cd2b6\") " pod="openshift-image-registry/node-ca-4h9s4" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041323 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87de50d6-3044-4202-bbee-c6e185f89313-tmp\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041335 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d3f60912-3413-4b9f-acb3-8d38dd276a82-konnectivity-ca\") pod \"konnectivity-agent-ggl9h\" (UID: \"d3f60912-3413-4b9f-acb3-8d38dd276a82\") " pod="kube-system/konnectivity-agent-ggl9h" Sep 06 10:30:09.043341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041370 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-systemd-units\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.044081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041502 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a28d14f7-334e-4b14-b75b-14619d4c4f4f-host-run-k8s-cni-cncf-io\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.044081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041548 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-var-lib-openvswitch\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.044081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041578 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-host-cni-netd\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.044081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041612 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/710a94f2-acb3-4a5e-98c8-e8aab73e1547-systemd-units\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.044081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.041800 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a28d14f7-334e-4b14-b75b-14619d4c4f4f-multus-daemon-config\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.044081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.042327 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d3f60912-3413-4b9f-acb3-8d38dd276a82-agent-certs\") pod \"konnectivity-agent-ggl9h\" (UID: \"d3f60912-3413-4b9f-acb3-8d38dd276a82\") " pod="kube-system/konnectivity-agent-ggl9h" Sep 06 10:30:09.044081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.042465 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/710a94f2-acb3-4a5e-98c8-e8aab73e1547-ovn-node-metrics-cert\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.044081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.043186 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87de50d6-3044-4202-bbee-c6e185f89313-etc-tuned\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.044081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.043818 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87de50d6-3044-4202-bbee-c6e185f89313-tmp\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.046546 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.046521 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lcxw\" (UniqueName: \"kubernetes.io/projected/41f8d38b-5235-4d1e-9840-2931fc5cd2b6-kube-api-access-4lcxw\") pod \"node-ca-4h9s4\" (UID: \"41f8d38b-5235-4d1e-9840-2931fc5cd2b6\") " pod="openshift-image-registry/node-ca-4h9s4" Sep 06 10:30:09.049086 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.049051 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 10:30:09.049086 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.049079 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 10:30:09.049282 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.049093 2574 projected.go:196] Error preparing data for projected volume kube-api-access-dpfh8 for pod openshift-network-diagnostics/network-check-target-lrpg9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:09.049282 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.049213 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8 podName:dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:09.549192631 +0000 UTC m=+3.131682244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dpfh8" (UniqueName: "kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8") pod "network-check-target-lrpg9" (UID: "dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:09.050366 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.050337 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q42k5\" (UniqueName: \"kubernetes.io/projected/87de50d6-3044-4202-bbee-c6e185f89313-kube-api-access-q42k5\") pod \"tuned-2hg8p\" (UID: \"87de50d6-3044-4202-bbee-c6e185f89313\") " pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.051224 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.051204 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhctk\" (UniqueName: \"kubernetes.io/projected/2da0a8a3-afbc-41fc-bd56-69500eb10934-kube-api-access-fhctk\") pod \"node-resolver-hn5z4\" (UID: \"2da0a8a3-afbc-41fc-bd56-69500eb10934\") " pod="openshift-dns/node-resolver-hn5z4" Sep 06 10:30:09.052145 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.052075 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g5bd\" (UniqueName: \"kubernetes.io/projected/a28d14f7-334e-4b14-b75b-14619d4c4f4f-kube-api-access-8g5bd\") pod \"multus-6rcdb\" (UID: \"a28d14f7-334e-4b14-b75b-14619d4c4f4f\") " pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.052995 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.052972 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp76q\" (UniqueName: \"kubernetes.io/projected/710a94f2-acb3-4a5e-98c8-e8aab73e1547-kube-api-access-gp76q\") pod \"ovnkube-node-cj6jp\" (UID: \"710a94f2-acb3-4a5e-98c8-e8aab73e1547\") " pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.055901 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.055884 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stgbv\" (UniqueName: \"kubernetes.io/projected/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-kube-api-access-stgbv\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:09.074700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.074672 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 10:30:09.131543 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.131512 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gfwjq" Sep 06 10:30:09.138405 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.138375 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" Sep 06 10:30:09.150235 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.150209 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hfgzs" Sep 06 10:30:09.155935 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.155916 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hn5z4" Sep 06 10:30:09.162497 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.162478 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4h9s4" Sep 06 10:30:09.169136 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.169106 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" Sep 06 10:30:09.176800 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.176783 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6rcdb" Sep 06 10:30:09.182436 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.182414 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:09.188137 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.188104 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ggl9h" Sep 06 10:30:09.321829 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.321751 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 06 10:30:09.444622 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.444593 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:09.444749 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.444714 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:09.444798 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.444766 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret podName:d037fa14-5fbf-4635-92df-3b0117254fca nodeName:}" failed. No retries permitted until 2026-09-06 10:30:10.44475099 +0000 UTC m=+4.027240598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret") pod "global-pull-secret-syncer-n424n" (UID: "d037fa14-5fbf-4635-92df-3b0117254fca") : object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:09.545245 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.545214 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:09.545413 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.545382 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:09.545465 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.545442 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs podName:23d754da-9be5-4fe4-9d31-0ce5f1c3b63e nodeName:}" failed. No retries permitted until 2026-09-06 10:30:10.545427093 +0000 UTC m=+4.127916729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs") pod "network-metrics-daemon-tg7pf" (UID: "23d754da-9be5-4fe4-9d31-0ce5f1c3b63e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:09.646140 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.646050 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpfh8\" (UniqueName: \"kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8\") pod \"network-check-target-lrpg9\" (UID: \"dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4\") " pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:09.646279 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.646236 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 10:30:09.646279 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.646256 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 10:30:09.646279 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.646269 2574 projected.go:196] Error preparing data for projected volume kube-api-access-dpfh8 for pod openshift-network-diagnostics/network-check-target-lrpg9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:09.646393 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.646321 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8 podName:dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:10.646307846 +0000 UTC m=+4.228797468 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dpfh8" (UniqueName: "kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8") pod "network-check-target-lrpg9" (UID: "dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:09.762229 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:09.762198 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2da0a8a3_afbc_41fc_bd56_69500eb10934.slice/crio-e1c46a416949f55cc181db0915311d4a3a8b7ea8325a040cf75d4003d7d4a879 WatchSource:0}: Error finding container e1c46a416949f55cc181db0915311d4a3a8b7ea8325a040cf75d4003d7d4a879: Status 404 returned error can't find the container with id e1c46a416949f55cc181db0915311d4a3a8b7ea8325a040cf75d4003d7d4a879 Sep 06 10:30:09.784341 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:09.784315 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f60912_3413_4b9f_acb3_8d38dd276a82.slice/crio-9da7e5697c199ed3ad23c0877c2a02b5ae947dee39911fbeac7516546190a350 WatchSource:0}: Error finding container 9da7e5697c199ed3ad23c0877c2a02b5ae947dee39911fbeac7516546190a350: Status 404 returned error can't find the container with id 9da7e5697c199ed3ad23c0877c2a02b5ae947dee39911fbeac7516546190a350 Sep 06 10:30:09.785062 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:09.785028 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5490127e_cffd_4c95_bccc_8709ee887aaf.slice/crio-6d83bced53d6583feeb8dc38aa2d6b333f51bfbe49da6950b3f9882b488efbad WatchSource:0}: Error finding container 6d83bced53d6583feeb8dc38aa2d6b333f51bfbe49da6950b3f9882b488efbad: Status 404 returned error can't find the container with id 6d83bced53d6583feeb8dc38aa2d6b333f51bfbe49da6950b3f9882b488efbad Sep 06 10:30:09.786767 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:09.786748 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52513a46_7d58_48a5_a6a9_7633b34482c7.slice/crio-316327773a815ef7b4bcdb3c70acab87fb6782d60340426acb918e2a39811097 WatchSource:0}: Error finding container 316327773a815ef7b4bcdb3c70acab87fb6782d60340426acb918e2a39811097: Status 404 returned error can't find the container with id 316327773a815ef7b4bcdb3c70acab87fb6782d60340426acb918e2a39811097 Sep 06 10:30:09.792622 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:09.792588 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda28d14f7_334e_4b14_b75b_14619d4c4f4f.slice/crio-6d935b59a020d8e217cb259168c2fe8871683439406b7fee356d90c0b7447994 WatchSource:0}: Error finding container 6d935b59a020d8e217cb259168c2fe8871683439406b7fee356d90c0b7447994: Status 404 returned error can't find the container with id 6d935b59a020d8e217cb259168c2fe8871683439406b7fee356d90c0b7447994 Sep 06 10:30:09.793586 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:09.793557 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f8d38b_5235_4d1e_9840_2931fc5cd2b6.slice/crio-8ad7fa71bdecaf3cd8423910b3221df810db8031b74475ff5ec54c6e1de67d2d WatchSource:0}: Error finding container 8ad7fa71bdecaf3cd8423910b3221df810db8031b74475ff5ec54c6e1de67d2d: Status 404 returned error can't find the container with id 8ad7fa71bdecaf3cd8423910b3221df810db8031b74475ff5ec54c6e1de67d2d Sep 06 10:30:09.794621 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:09.794592 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87de50d6_3044_4202_bbee_c6e185f89313.slice/crio-35287588aa0cfc25a89e732a0ee6a1da68dc405bc1fac564eb568bba01faf3f0 WatchSource:0}: Error finding container 35287588aa0cfc25a89e732a0ee6a1da68dc405bc1fac564eb568bba01faf3f0: Status 404 returned error can't find the container with id 35287588aa0cfc25a89e732a0ee6a1da68dc405bc1fac564eb568bba01faf3f0 Sep 06 10:30:09.865378 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.865332 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 10:25:08 +0000 UTC" deadline="2028-04-04 16:28:19.203725317 +0000 UTC" Sep 06 10:30:09.865964 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.865379 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13829h58m9.338350051s" Sep 06 10:30:09.920810 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.920736 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:09.920938 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:09.920839 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:09.932329 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.932294 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-139.ec2.internal" event={"ID":"d442c7b86c5fc53cff3e2ee8151b31bc","Type":"ContainerStarted","Data":"8b8b18296f2186dda3dec22e5697fe26033e33511f79fb7f3d63ca48051a2ed0"} Sep 06 10:30:09.936154 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.936114 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" event={"ID":"710a94f2-acb3-4a5e-98c8-e8aab73e1547","Type":"ContainerStarted","Data":"38e45081e1e3e4d93b871886fcd632dc6ddb3deb6a8fe71bb12856883f7537c6"} Sep 06 10:30:09.938377 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.938357 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gfwjq" event={"ID":"85ed5170-128c-493b-8737-94af607505c6","Type":"ContainerStarted","Data":"4713854baf90691ae6ee79ffb363b813c199bc802af1103d05dc656b5d40768c"} Sep 06 10:30:09.940632 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.940611 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" event={"ID":"87de50d6-3044-4202-bbee-c6e185f89313","Type":"ContainerStarted","Data":"35287588aa0cfc25a89e732a0ee6a1da68dc405bc1fac564eb568bba01faf3f0"} Sep 06 10:30:09.943239 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.943202 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6rcdb" event={"ID":"a28d14f7-334e-4b14-b75b-14619d4c4f4f","Type":"ContainerStarted","Data":"6d935b59a020d8e217cb259168c2fe8871683439406b7fee356d90c0b7447994"} Sep 06 10:30:09.944937 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.944915 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" event={"ID":"5490127e-cffd-4c95-bccc-8709ee887aaf","Type":"ContainerStarted","Data":"6d83bced53d6583feeb8dc38aa2d6b333f51bfbe49da6950b3f9882b488efbad"} Sep 06 10:30:09.945214 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.945169 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-139.ec2.internal" podStartSLOduration=2.9451587569999997 podStartE2EDuration="2.945158757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:09.945136845 +0000 UTC m=+3.527626468" watchObservedRunningTime="2026-09-06 10:30:09.945158757 +0000 UTC m=+3.527648387" Sep 06 10:30:09.946026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.946006 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ggl9h" event={"ID":"d3f60912-3413-4b9f-acb3-8d38dd276a82","Type":"ContainerStarted","Data":"9da7e5697c199ed3ad23c0877c2a02b5ae947dee39911fbeac7516546190a350"} Sep 06 10:30:09.947835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.947809 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4h9s4" event={"ID":"41f8d38b-5235-4d1e-9840-2931fc5cd2b6","Type":"ContainerStarted","Data":"8ad7fa71bdecaf3cd8423910b3221df810db8031b74475ff5ec54c6e1de67d2d"} Sep 06 10:30:09.949861 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.949832 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hfgzs" event={"ID":"52513a46-7d58-48a5-a6a9-7633b34482c7","Type":"ContainerStarted","Data":"316327773a815ef7b4bcdb3c70acab87fb6782d60340426acb918e2a39811097"} Sep 06 10:30:09.951551 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:09.951533 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hn5z4" event={"ID":"2da0a8a3-afbc-41fc-bd56-69500eb10934","Type":"ContainerStarted","Data":"e1c46a416949f55cc181db0915311d4a3a8b7ea8325a040cf75d4003d7d4a879"} Sep 06 10:30:10.450570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:10.450534 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:10.450761 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:10.450706 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:10.450830 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:10.450795 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret podName:d037fa14-5fbf-4635-92df-3b0117254fca nodeName:}" failed. No retries permitted until 2026-09-06 10:30:12.450770701 +0000 UTC m=+6.033260324 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret") pod "global-pull-secret-syncer-n424n" (UID: "d037fa14-5fbf-4635-92df-3b0117254fca") : object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:10.551752 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:10.551711 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:10.551913 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:10.551860 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:10.551981 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:10.551922 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs podName:23d754da-9be5-4fe4-9d31-0ce5f1c3b63e nodeName:}" failed. No retries permitted until 2026-09-06 10:30:12.55190431 +0000 UTC m=+6.134393932 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs") pod "network-metrics-daemon-tg7pf" (UID: "23d754da-9be5-4fe4-9d31-0ce5f1c3b63e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:10.652307 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:10.652271 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpfh8\" (UniqueName: \"kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8\") pod \"network-check-target-lrpg9\" (UID: \"dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4\") " pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:10.652487 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:10.652471 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 10:30:10.652542 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:10.652491 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 10:30:10.652542 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:10.652504 2574 projected.go:196] Error preparing data for projected volume kube-api-access-dpfh8 for pod openshift-network-diagnostics/network-check-target-lrpg9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:10.652645 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:10.652562 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8 podName:dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:12.652545049 +0000 UTC m=+6.235034659 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dpfh8" (UniqueName: "kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8") pod "network-check-target-lrpg9" (UID: "dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:10.865974 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:10.865878 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 10:25:08 +0000 UTC" deadline="2028-02-13 20:47:25.422556198 +0000 UTC" Sep 06 10:30:10.865974 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:10.865932 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12610h17m14.556639461s" Sep 06 10:30:10.923077 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:10.923047 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:10.923260 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:10.923214 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:10.923641 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:10.923622 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:10.923739 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:10.923717 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:10.959566 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:10.959530 2574 generic.go:356] "Generic (PLEG): container finished" podID="03d807d2b2681f19d290cbd8bf76f7e7" containerID="d82884be29ce78b717ce4bbb9bb2793a2ea9cb317a13f9de5c6a6eed5e478451" exitCode=0 Sep 06 10:30:10.960235 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:10.960209 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" event={"ID":"03d807d2b2681f19d290cbd8bf76f7e7","Type":"ContainerDied","Data":"d82884be29ce78b717ce4bbb9bb2793a2ea9cb317a13f9de5c6a6eed5e478451"} Sep 06 10:30:11.921137 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:11.920818 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:11.921601 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:11.921193 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:11.988776 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:11.988637 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" event={"ID":"03d807d2b2681f19d290cbd8bf76f7e7","Type":"ContainerStarted","Data":"f759a3ad33276b157442138d03939c63fe2a428cd801040d3c9f2b7b34f526f0"} Sep 06 10:30:12.470247 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:12.470148 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:12.470410 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:12.470289 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:12.470410 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:12.470351 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret podName:d037fa14-5fbf-4635-92df-3b0117254fca nodeName:}" failed. No retries permitted until 2026-09-06 10:30:16.470332293 +0000 UTC m=+10.052821918 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret") pod "global-pull-secret-syncer-n424n" (UID: "d037fa14-5fbf-4635-92df-3b0117254fca") : object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:12.574159 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:12.570562 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:12.574159 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:12.570709 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:12.574159 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:12.570777 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs podName:23d754da-9be5-4fe4-9d31-0ce5f1c3b63e nodeName:}" failed. No retries permitted until 2026-09-06 10:30:16.570758205 +0000 UTC m=+10.153247839 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs") pod "network-metrics-daemon-tg7pf" (UID: "23d754da-9be5-4fe4-9d31-0ce5f1c3b63e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:12.671913 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:12.671875 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpfh8\" (UniqueName: \"kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8\") pod \"network-check-target-lrpg9\" (UID: \"dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4\") " pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:12.672095 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:12.672074 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 10:30:12.672190 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:12.672098 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 10:30:12.672190 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:12.672110 2574 projected.go:196] Error preparing data for projected volume kube-api-access-dpfh8 for pod openshift-network-diagnostics/network-check-target-lrpg9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:12.672190 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:12.672184 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8 podName:dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:16.672166071 +0000 UTC m=+10.254655682 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dpfh8" (UniqueName: "kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8") pod "network-check-target-lrpg9" (UID: "dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:12.921599 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:12.921100 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:12.922053 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:12.921977 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:12.922175 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:12.922136 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:12.923074 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:12.923048 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:13.920662 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:13.920203 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:13.920662 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:13.920321 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:14.920301 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:14.920247 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:14.920301 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:14.920299 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:14.920792 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:14.920432 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:14.920792 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:14.920540 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:15.921638 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:15.921075 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:15.921638 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:15.921220 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:16.505842 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:16.505800 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:16.506030 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:16.505973 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:16.506096 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:16.506052 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret podName:d037fa14-5fbf-4635-92df-3b0117254fca nodeName:}" failed. No retries permitted until 2026-09-06 10:30:24.506030204 +0000 UTC m=+18.088519831 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret") pod "global-pull-secret-syncer-n424n" (UID: "d037fa14-5fbf-4635-92df-3b0117254fca") : object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:16.606933 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:16.606862 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:16.607103 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:16.607008 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:16.607103 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:16.607084 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs podName:23d754da-9be5-4fe4-9d31-0ce5f1c3b63e nodeName:}" failed. No retries permitted until 2026-09-06 10:30:24.607062708 +0000 UTC m=+18.189552333 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs") pod "network-metrics-daemon-tg7pf" (UID: "23d754da-9be5-4fe4-9d31-0ce5f1c3b63e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:16.708081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:16.708042 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpfh8\" (UniqueName: \"kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8\") pod \"network-check-target-lrpg9\" (UID: \"dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4\") " pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:16.708253 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:16.708214 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 10:30:16.708253 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:16.708236 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 10:30:16.708253 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:16.708250 2574 projected.go:196] Error preparing data for projected volume kube-api-access-dpfh8 for pod openshift-network-diagnostics/network-check-target-lrpg9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:16.708370 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:16.708318 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8 podName:dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:24.708299008 +0000 UTC m=+18.290788636 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dpfh8" (UniqueName: "kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8") pod "network-check-target-lrpg9" (UID: "dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:16.929200 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:16.927176 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:16.929200 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:16.927321 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:16.929200 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:16.927442 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:16.929200 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:16.927569 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:17.920387 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:17.920284 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:17.920561 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:17.920420 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:18.920810 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:18.920779 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:18.921362 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:18.920893 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:18.921362 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:18.920983 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:18.921362 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:18.921100 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:19.920958 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:19.920928 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:19.921345 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:19.921032 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:20.922974 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:20.922943 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:20.922974 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:20.922986 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:20.923483 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:20.923074 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:20.923483 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:20.923243 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:21.921043 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:21.921002 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:21.921245 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:21.921146 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:22.923360 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:22.923330 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:22.923871 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:22.923337 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:22.923871 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:22.923490 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:22.923871 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:22.923527 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:23.920941 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:23.920905 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:23.921182 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:23.921026 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:24.569901 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:24.569860 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:24.570336 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:24.570005 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:24.570336 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:24.570078 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret podName:d037fa14-5fbf-4635-92df-3b0117254fca nodeName:}" failed. No retries permitted until 2026-09-06 10:30:40.570057748 +0000 UTC m=+34.152547383 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret") pod "global-pull-secret-syncer-n424n" (UID: "d037fa14-5fbf-4635-92df-3b0117254fca") : object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:24.670827 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:24.670783 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:24.671018 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:24.670920 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:24.671018 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:24.670997 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs podName:23d754da-9be5-4fe4-9d31-0ce5f1c3b63e nodeName:}" failed. No retries permitted until 2026-09-06 10:30:40.670976748 +0000 UTC m=+34.253466382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs") pod "network-metrics-daemon-tg7pf" (UID: "23d754da-9be5-4fe4-9d31-0ce5f1c3b63e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:24.771555 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:24.771524 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpfh8\" (UniqueName: \"kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8\") pod \"network-check-target-lrpg9\" (UID: \"dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4\") " pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:24.771721 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:24.771654 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 10:30:24.771721 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:24.771668 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 10:30:24.771721 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:24.771677 2574 projected.go:196] Error preparing data for projected volume kube-api-access-dpfh8 for pod openshift-network-diagnostics/network-check-target-lrpg9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:24.771903 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:24.771728 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8 podName:dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4 nodeName:}" failed. No retries permitted until 2026-09-06 10:30:40.771712414 +0000 UTC m=+34.354202045 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dpfh8" (UniqueName: "kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8") pod "network-check-target-lrpg9" (UID: "dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:24.921053 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:24.920981 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:24.921215 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:24.920984 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:24.921215 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:24.921119 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:24.921215 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:24.921189 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:25.920605 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:25.920564 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:25.921027 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:25.920677 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:26.920948 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:26.920916 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:26.921438 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:26.921007 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:26.921438 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:26.921057 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:26.921438 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:26.921155 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:27.920752 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:27.920603 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:27.920848 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:27.920825 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:28.021648 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.021615 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" event={"ID":"710a94f2-acb3-4a5e-98c8-e8aab73e1547","Type":"ContainerStarted","Data":"f1f86ed2c42a4680ae4aef1aed89884e76be13742688c3590b5e7a1aa5da5653"} Sep 06 10:30:28.022446 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.021690 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" event={"ID":"710a94f2-acb3-4a5e-98c8-e8aab73e1547","Type":"ContainerStarted","Data":"3daeb0fe26c13f3758b196ebed5dffbc73aa6f737d0181819092a2f431751d97"} Sep 06 10:30:28.023174 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.023145 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" event={"ID":"87de50d6-3044-4202-bbee-c6e185f89313","Type":"ContainerStarted","Data":"d29e1f634a96f2e88a53a6f220be866e61eb9b3fe7d4eab48b5a3f1180181707"} Sep 06 10:30:28.024643 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.024623 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6rcdb" event={"ID":"a28d14f7-334e-4b14-b75b-14619d4c4f4f","Type":"ContainerStarted","Data":"5f2467a779809e1dcfe5074e3226e1b61cfd716a504a9955e3e4dc1b988de967"} Sep 06 10:30:28.026046 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.026024 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" event={"ID":"5490127e-cffd-4c95-bccc-8709ee887aaf","Type":"ContainerStarted","Data":"3e89d14d165b57b22b99f415bbc0318f2b3ec8aec1b7b0d6890ff36c50cd82a5"} Sep 06 10:30:28.027549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.027529 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ggl9h" event={"ID":"d3f60912-3413-4b9f-acb3-8d38dd276a82","Type":"ContainerStarted","Data":"bb6b6cdeff97be3d87460eb17cf38752c7cb607ee6cfab755f44e5d7557d3b02"} Sep 06 10:30:28.028812 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.028793 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4h9s4" event={"ID":"41f8d38b-5235-4d1e-9840-2931fc5cd2b6","Type":"ContainerStarted","Data":"38854d47065d8aed6e5d08f4786f7bed31763ed33415d98c8556e3173b8d6611"} Sep 06 10:30:28.030204 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.030183 2574 generic.go:356] "Generic (PLEG): container finished" podID="52513a46-7d58-48a5-a6a9-7633b34482c7" containerID="1001ed785599517a7059cd36bd7e5cbc7be1218fe647a87f70416b199a269a89" exitCode=0 Sep 06 10:30:28.030293 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.030246 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hfgzs" event={"ID":"52513a46-7d58-48a5-a6a9-7633b34482c7","Type":"ContainerDied","Data":"1001ed785599517a7059cd36bd7e5cbc7be1218fe647a87f70416b199a269a89"} Sep 06 10:30:28.031894 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.031876 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hn5z4" event={"ID":"2da0a8a3-afbc-41fc-bd56-69500eb10934","Type":"ContainerStarted","Data":"667906d0c8296d16a4bf82262c71dbccb7a90a4b6bfe4917b8bbd041900bdba9"} Sep 06 10:30:28.039723 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.039685 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2hg8p" podStartSLOduration=3.306763816 podStartE2EDuration="21.039672457s" totalImagesPullingTime="17.732908641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:28.039436143 +0000 UTC m=+21.621925776" watchObservedRunningTime="2026-09-06 10:30:28.039672457 +0000 UTC m=+21.622162088" Sep 06 10:30:28.040311 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.040270 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-139.ec2.internal" podStartSLOduration=21.040258574 podStartE2EDuration="21.040258574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:12.004111087 +0000 UTC m=+5.586600719" watchObservedRunningTime="2026-09-06 10:30:28.040258574 +0000 UTC m=+21.622748208" Sep 06 10:30:28.073307 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.073183 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6rcdb" podStartSLOduration=3.294386168 podStartE2EDuration="21.073165437s" totalImagesPullingTime="17.778779269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:28.072890766 +0000 UTC m=+21.655380419" watchObservedRunningTime="2026-09-06 10:30:28.073165437 +0000 UTC m=+21.655655072" Sep 06 10:30:28.073680 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.073643 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hn5z4" podStartSLOduration=3.325110289 podStartE2EDuration="21.073632869s" totalImagesPullingTime="17.74852258s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:28.056232372 +0000 UTC m=+21.638722004" watchObservedRunningTime="2026-09-06 10:30:28.073632869 +0000 UTC m=+21.656122500" Sep 06 10:30:28.087544 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.087509 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4h9s4" podStartSLOduration=3.357747402 podStartE2EDuration="21.087496096s" totalImagesPullingTime="17.729748694s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:28.087024532 +0000 UTC m=+21.669514163" watchObservedRunningTime="2026-09-06 10:30:28.087496096 +0000 UTC m=+21.669985726" Sep 06 10:30:28.102041 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.102004 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ggl9h" podStartSLOduration=7.530438352 podStartE2EDuration="21.101990562s" totalImagesPullingTime="13.57155221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:28.101961769 +0000 UTC m=+21.684451401" watchObservedRunningTime="2026-09-06 10:30:28.101990562 +0000 UTC m=+21.684480171" Sep 06 10:30:28.193692 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.193662 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:28.271081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.271044 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:28.920692 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.920664 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:28.920692 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.920681 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:28.920862 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:28.920834 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:28.920961 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:28.920935 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:28.933640 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:28.933620 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 06 10:30:29.038141 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:29.038093 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" event={"ID":"710a94f2-acb3-4a5e-98c8-e8aab73e1547","Type":"ContainerStarted","Data":"cf312d8a6c8ba79bcb883f74588fde321d210fc4018c7ac2945462a70858f4a3"} Sep 06 10:30:29.038569 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:29.038203 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" event={"ID":"710a94f2-acb3-4a5e-98c8-e8aab73e1547","Type":"ContainerStarted","Data":"6a8d7531951c2ed8231684cdf8ad737e5bdc2929e613100b3146462a9c29d71d"} Sep 06 10:30:29.038569 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:29.038250 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" event={"ID":"710a94f2-acb3-4a5e-98c8-e8aab73e1547","Type":"ContainerStarted","Data":"5a65f152a18434ef5baa419f0e219793d6dac100d7e3de65d43e938e1bd711c3"} Sep 06 10:30:29.038569 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:29.038297 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" event={"ID":"710a94f2-acb3-4a5e-98c8-e8aab73e1547","Type":"ContainerStarted","Data":"730e3d47d4b423569789e7aa13d950d32994bafe5b246167bef664336c04f959"} Sep 06 10:30:29.040310 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:29.040271 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" event={"ID":"5490127e-cffd-4c95-bccc-8709ee887aaf","Type":"ContainerStarted","Data":"b3617e633c1bea575896afc8280a3d535a25c31356cd2a02863204c74c5073fa"} Sep 06 10:30:29.890724 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:29.890620 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-06T10:30:28.933635718Z","UUID":"f940203d-26b4-4386-a976-db96fcea2487","Handler":null,"Name":"","Endpoint":""} Sep 06 10:30:29.892194 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:29.892170 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 06 10:30:29.892315 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:29.892201 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 06 10:30:29.920351 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:29.920318 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:29.920486 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:29.920433 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:30.044803 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:30.044763 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gfwjq" event={"ID":"85ed5170-128c-493b-8737-94af607505c6","Type":"ContainerStarted","Data":"3fcd34e29e872ec3a9b85f99556f8c9bae1ae8c3bd91751d2a2d48d1792723d0"} Sep 06 10:30:30.058227 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:30.058176 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gfwjq" podStartSLOduration=6.329943777 podStartE2EDuration="24.058154545s" totalImagesPullingTime="17.728210768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:30.058023766 +0000 UTC m=+23.640513398" watchObservedRunningTime="2026-09-06 10:30:30.058154545 +0000 UTC m=+23.640644177" Sep 06 10:30:30.275801 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:30.275758 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:30.347391 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:30.347349 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:30.769831 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:30.769750 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ggl9h" Sep 06 10:30:30.770446 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:30.770418 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ggl9h" Sep 06 10:30:30.921048 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:30.921012 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:30.921246 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:30.921013 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:30.921246 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:30.921178 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:30.921370 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:30.921258 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:31.049271 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:31.049180 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" event={"ID":"5490127e-cffd-4c95-bccc-8709ee887aaf","Type":"ContainerStarted","Data":"faa92d48d372df38d9de37277b387840880b255ae4444a19e97d9d5faa1c657b"} Sep 06 10:30:31.053293 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:31.053248 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" event={"ID":"710a94f2-acb3-4a5e-98c8-e8aab73e1547","Type":"ContainerStarted","Data":"a6b5c722998167a0fc6807f6f0842311c16695c16f4697beda7802977f3d0069"} Sep 06 10:30:31.920813 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:31.920782 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:31.921009 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:31.920904 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:32.451301 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:32.451272 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:32.555701 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:32.555470 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:32.920750 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:32.920712 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:32.920889 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:32.920851 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:32.920944 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:32.920892 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:32.921007 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:32.920990 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:33.000106 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.000032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:33.061250 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.061218 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" event={"ID":"710a94f2-acb3-4a5e-98c8-e8aab73e1547","Type":"ContainerStarted","Data":"6b95e279bd5a3a3adb7b6fbc38c6cfd2b6ebece303a560cbfbc76ce388a8a4fa"} Sep 06 10:30:33.061508 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.061486 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:33.061635 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.061561 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:33.063208 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.063180 2574 generic.go:356] "Generic (PLEG): container finished" podID="52513a46-7d58-48a5-a6a9-7633b34482c7" containerID="c9500ab489d5603d65bd11d7dd3cb5b6b722a2d7a1c3a4b72660bc75d7210b7e" exitCode=0 Sep 06 10:30:33.063208 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.063207 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hfgzs" event={"ID":"52513a46-7d58-48a5-a6a9-7633b34482c7","Type":"ContainerDied","Data":"c9500ab489d5603d65bd11d7dd3cb5b6b722a2d7a1c3a4b72660bc75d7210b7e"} Sep 06 10:30:33.070684 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.070658 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:33.077468 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.077445 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:33.088457 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.088399 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrlbx" podStartSLOduration=7.209458715 podStartE2EDuration="27.088381164s" totalImagesPullingTime="19.878922449s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:31.065470808 +0000 UTC m=+24.647960452" watchObservedRunningTime="2026-09-06 10:30:33.088381164 +0000 UTC m=+26.670870801" Sep 06 10:30:33.113203 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.113161 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" podStartSLOduration=8.235507371 podStartE2EDuration="26.113148445s" totalImagesPullingTime="17.877641074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:33.090342631 +0000 UTC m=+26.672832255" watchObservedRunningTime="2026-09-06 10:30:33.113148445 +0000 UTC m=+26.695638073" Sep 06 10:30:33.133358 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.133335 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:33.582875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.582837 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:33.635955 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.635922 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:33.834452 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.834372 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:33.921157 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.921110 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:33.921323 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:33.921274 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:33.934692 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:33.934659 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:34.066491 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:34.066449 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:34.080706 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:34.080682 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:30:34.545947 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:34.545657 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lrpg9"] Sep 06 10:30:34.546105 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:34.545954 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:34.546105 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:34.546039 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:34.548227 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:34.548198 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n424n"] Sep 06 10:30:34.548334 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:34.548301 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:34.548439 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:34.548418 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:34.548985 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:34.548962 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tg7pf"] Sep 06 10:30:34.549108 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:34.549078 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:34.549221 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:34.549200 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:35.070248 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:35.070216 2574 generic.go:356] "Generic (PLEG): container finished" podID="52513a46-7d58-48a5-a6a9-7633b34482c7" containerID="3e794b10e525d98fc134bfd19370a7ef61039c1b6e689e3eaa1b149cd97cd8f9" exitCode=0 Sep 06 10:30:35.070612 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:35.070295 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hfgzs" event={"ID":"52513a46-7d58-48a5-a6a9-7633b34482c7","Type":"ContainerDied","Data":"3e794b10e525d98fc134bfd19370a7ef61039c1b6e689e3eaa1b149cd97cd8f9"} Sep 06 10:30:35.125721 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:35.125685 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ggl9h" Sep 06 10:30:35.125886 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:35.125819 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 06 10:30:35.126278 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:35.126260 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ggl9h" Sep 06 10:30:35.181454 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:35.181407 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:35.404966 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:35.404891 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:35.463781 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:35.463749 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:35.531079 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:35.531050 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hn5z4_2da0a8a3-afbc-41fc-bd56-69500eb10934/dns-node-resolver/0.log" Sep 06 10:30:35.920298 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:35.920265 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:35.920454 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:35.920309 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:35.920454 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:35.920359 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:35.920454 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:35.920372 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:35.920454 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:35.920439 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:35.920574 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:35.920493 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:36.301968 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:36.301894 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:36.331438 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:36.331416 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4h9s4_41f8d38b-5235-4d1e-9840-2931fc5cd2b6/node-ca/0.log" Sep 06 10:30:37.077885 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:37.077820 2574 generic.go:356] "Generic (PLEG): container finished" podID="52513a46-7d58-48a5-a6a9-7633b34482c7" containerID="62975f0c62c1d735a296615e64872034106269f5f5981abb17381eb9fb1c0f37" exitCode=0 Sep 06 10:30:37.077885 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:37.077881 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hfgzs" event={"ID":"52513a46-7d58-48a5-a6a9-7633b34482c7","Type":"ContainerDied","Data":"62975f0c62c1d735a296615e64872034106269f5f5981abb17381eb9fb1c0f37"} Sep 06 10:30:37.831912 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:37.831832 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:37.921007 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:37.920972 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:37.921211 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:37.921013 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:37.921211 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:37.921013 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:37.921211 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:37.921103 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:37.921211 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:37.921188 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:37.921430 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:37.921256 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:38.685614 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:38.685566 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:39.920478 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:39.920437 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:39.920478 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:39.920478 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:39.920984 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:39.920412 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:39.920984 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:39.920557 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:39.920984 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:39.920614 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:39.920984 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:39.920685 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:40.444870 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:40.444834 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:40.527606 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:40.527559 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:40.583690 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:40.583654 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:40.583846 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:40.583805 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:40.583923 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:40.583878 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret podName:d037fa14-5fbf-4635-92df-3b0117254fca nodeName:}" failed. No retries permitted until 2026-09-06 10:31:12.583855785 +0000 UTC m=+66.166345395 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret") pod "global-pull-secret-syncer-n424n" (UID: "d037fa14-5fbf-4635-92df-3b0117254fca") : object "kube-system"/"original-pull-secret" not registered Sep 06 10:30:40.684942 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:40.684913 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:40.685099 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:40.685057 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:40.685151 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:40.685145 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs podName:23d754da-9be5-4fe4-9d31-0ce5f1c3b63e nodeName:}" failed. No retries permitted until 2026-09-06 10:31:12.685109576 +0000 UTC m=+66.267599192 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs") pod "network-metrics-daemon-tg7pf" (UID: "23d754da-9be5-4fe4-9d31-0ce5f1c3b63e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 06 10:30:40.785619 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:40.785536 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpfh8\" (UniqueName: \"kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8\") pod \"network-check-target-lrpg9\" (UID: \"dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4\") " pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:40.785762 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:40.785701 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 06 10:30:40.785762 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:40.785722 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 06 10:30:40.785762 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:40.785732 2574 projected.go:196] Error preparing data for projected volume kube-api-access-dpfh8 for pod openshift-network-diagnostics/network-check-target-lrpg9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:40.785899 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:40.785781 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8 podName:dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4 nodeName:}" failed. No retries permitted until 2026-09-06 10:31:12.785767854 +0000 UTC m=+66.368257463 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dpfh8" (UniqueName: "kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8") pod "network-check-target-lrpg9" (UID: "dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 06 10:30:41.920236 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:41.920205 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:41.920619 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:41.920296 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:41.920619 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:41.920317 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:41.920619 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:41.920302 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:41.920619 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:41.920389 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:41.920619 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:41.920442 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:42.093928 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:42.093898 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:42.199969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:42.199939 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:42.295034 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:42.294992 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:42.395168 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:42.395114 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:42.462525 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:42.462448 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:42.518900 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:42.518872 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:42.998134 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:42.998083 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:43.093983 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:43.093953 2574 generic.go:356] "Generic (PLEG): container finished" podID="52513a46-7d58-48a5-a6a9-7633b34482c7" containerID="3f1d4d30be57f20efe37d5402cc467165891cf923cc6165c2e5c60684ef8e9d2" exitCode=0 Sep 06 10:30:43.094148 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:43.093994 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hfgzs" event={"ID":"52513a46-7d58-48a5-a6a9-7633b34482c7","Type":"ContainerDied","Data":"3f1d4d30be57f20efe37d5402cc467165891cf923cc6165c2e5c60684ef8e9d2"} Sep 06 10:30:43.635997 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:43.635952 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:43.920698 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:43.920605 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:43.920839 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:43.920628 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:43.920839 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:43.920722 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:43.920839 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:43.920628 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:43.920839 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:43.920800 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:43.920969 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:43.920861 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:44.099556 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:44.099524 2574 generic.go:356] "Generic (PLEG): container finished" podID="52513a46-7d58-48a5-a6a9-7633b34482c7" containerID="58a9f4d34bf062ee6c5b48b509312c9a52b3ac0ddf957d53aa207fa23abafd72" exitCode=0 Sep 06 10:30:44.099962 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:44.099567 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hfgzs" event={"ID":"52513a46-7d58-48a5-a6a9-7633b34482c7","Type":"ContainerDied","Data":"58a9f4d34bf062ee6c5b48b509312c9a52b3ac0ddf957d53aa207fa23abafd72"} Sep 06 10:30:44.196686 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:44.196662 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:44.394826 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:44.394797 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:44.798704 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:44.798673 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:44.900183 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:44.900090 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:44.954347 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:44.954315 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:45.012364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:45.012331 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:45.104868 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:45.104837 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hfgzs" event={"ID":"52513a46-7d58-48a5-a6a9-7633b34482c7","Type":"ContainerStarted","Data":"ddca2b380892aceaf5df96601b9ea422d47ee4a94bc17e2afb8c24df66066f5b"} Sep 06 10:30:45.124257 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:45.124215 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hfgzs" podStartSLOduration=8.523388094 podStartE2EDuration="38.124200073s" totalImagesPullingTime="29.600811979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:30:45.123718701 +0000 UTC m=+38.706208357" watchObservedRunningTime="2026-09-06 10:30:45.124200073 +0000 UTC m=+38.706689704" Sep 06 10:30:45.401746 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:45.401712 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:45.920082 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:45.920049 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:45.920250 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:45.920048 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:45.920250 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:45.920183 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:45.920250 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:45.920214 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:45.920347 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:45.920294 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:45.920382 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:45.920369 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:45.993068 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:45.993039 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:46.047299 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:46.047271 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:46.396012 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:46.395927 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:46.805643 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:46.805565 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:46.859618 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:46.859597 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:47.194090 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:47.194015 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:47.344934 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:47.344902 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:47.405082 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:47.405051 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:47.795796 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:47.795761 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:47.921058 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:47.921022 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:47.921222 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:47.921022 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:47.921222 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:47.921163 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:47.921222 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:47.921022 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:47.921361 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:47.921230 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:47.921361 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:47.921274 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:48.192242 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:48.192157 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:48.255293 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:48.255252 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:49.013695 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:49.013658 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:49.196753 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:49.196721 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:49.603653 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:49.603624 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:49.794961 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:49.794933 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:49.920544 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:49.920464 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:49.920700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:49.920464 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:49.920700 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:49.920587 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:49.920700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:49.920464 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:49.920700 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:49.920666 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:49.920898 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:49.920769 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:50.403041 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:50.402963 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:50.992274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:50.992244 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:51.192215 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:51.192177 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:51.394904 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:51.394829 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:51.796736 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:51.796658 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:51.920511 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:51.920476 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:51.920697 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:51.920522 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:51.920697 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:51.920597 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:51.920697 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:51.920664 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:51.920856 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:51.920713 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:51.920856 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:51.920799 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:52.196360 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:52.196275 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:52.620888 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:52.620845 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:52.793035 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:52.793006 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:53.920700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:53.920665 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:53.921093 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:53.920668 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:53.921093 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:53.920766 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:53.921093 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:53.920870 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:53.921093 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:53.920668 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:53.921093 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:53.920950 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:55.422775 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:55.422744 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:55.483189 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:55.483158 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:55.920667 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:55.920646 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:55.920808 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:55.920731 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:55.920849 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:55.920804 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:55.920895 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:55.920878 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:55.920933 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:55.920918 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:55.920987 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:55.920975 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:57.400154 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:57.400106 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:57.503350 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:57.503313 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:57.607947 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:57.607912 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:57.803022 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:57.802990 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:57.865329 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:57.865297 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:57.920937 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:57.920901 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:57.920937 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:57.920944 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:57.921156 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:57.920948 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:57.921156 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:57.921022 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrpg9" podUID="dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4" Sep 06 10:30:57.921156 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:57.921098 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n424n" podUID="d037fa14-5fbf-4635-92df-3b0117254fca" Sep 06 10:30:57.921266 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:30:57.921196 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tg7pf" podUID="23d754da-9be5-4fe4-9d31-0ce5f1c3b63e" Sep 06 10:30:58.754736 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.754657 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-139.ec2.internal" event="NodeReady" Sep 06 10:30:58.755176 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.754782 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 06 10:30:58.793880 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.793838 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66799f8b9-wfgqs"] Sep 06 10:30:58.812477 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.812450 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bdp7c"] Sep 06 10:30:58.812631 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.812587 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:58.814922 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.814897 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 06 10:30:58.815057 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.814924 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 06 10:30:58.815057 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.814944 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 06 10:30:58.815057 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.814944 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f2fgs\"" Sep 06 10:30:58.823170 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.823141 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 06 10:30:58.830333 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.830309 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-56h9s"] Sep 06 10:30:58.830487 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.830440 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:58.832685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.832662 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 06 10:30:58.832812 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.832703 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-89vqh\"" Sep 06 10:30:58.832812 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.832662 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 06 10:30:58.847978 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.847949 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66799f8b9-wfgqs"] Sep 06 10:30:58.847978 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.847981 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-56h9s"] Sep 06 10:30:58.848164 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.848003 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bdp7c"] Sep 06 10:30:58.848164 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.848084 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:58.850689 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.850668 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 06 10:30:58.850789 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.850673 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 06 10:30:58.850789 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.850725 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 06 10:30:58.850789 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.850749 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l4ftj\"" Sep 06 10:30:58.850789 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.850752 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 06 10:30:58.908986 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.908948 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rh5rj"] Sep 06 10:30:58.912191 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.912171 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1-metrics-tls\") pod \"dns-default-bdp7c\" (UID: \"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1\") " pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:58.912266 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.912201 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-registry-tls\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:58.912266 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.912227 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-ca-trust-extracted\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:58.912266 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.912256 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1-config-volume\") pod \"dns-default-bdp7c\" (UID: \"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1\") " pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:58.912363 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.912274 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-trusted-ca\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:58.912363 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.912291 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpnqk\" (UniqueName: \"kubernetes.io/projected/a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1-kube-api-access-xpnqk\") pod \"dns-default-bdp7c\" (UID: \"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1\") " pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:58.912363 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.912305 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6c5p\" (UniqueName: \"kubernetes.io/projected/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-kube-api-access-h6c5p\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:58.912363 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.912330 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1-tmp-dir\") pod \"dns-default-bdp7c\" (UID: \"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1\") " pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:58.912363 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.912353 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-image-registry-private-configuration\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:58.912507 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.912412 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-installation-pull-secrets\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:58.912507 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.912439 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-registry-certificates\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:58.912507 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.912452 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-bound-sa-token\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:58.934482 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.934452 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rh5rj" Sep 06 10:30:58.934658 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.934636 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:30:58.936924 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.936896 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 06 10:30:58.937058 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.936928 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bx9l2\"" Sep 06 10:30:58.937058 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.937032 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-66txg\"" Sep 06 10:30:58.937201 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.937066 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 06 10:30:58.937201 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.937106 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 06 10:30:58.937482 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.937466 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 06 10:30:58.938001 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.937985 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rh5rj"] Sep 06 10:30:58.942158 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:58.942136 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:59.013861 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.013773 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1-metrics-tls\") pod \"dns-default-bdp7c\" (UID: \"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1\") " pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:59.013861 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.013809 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-registry-tls\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.013861 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.013838 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-crio-socket\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.014146 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.013868 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-ca-trust-extracted\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.014146 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.013891 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bq7d\" (UniqueName: \"kubernetes.io/projected/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-kube-api-access-2bq7d\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.014146 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.013929 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1-config-volume\") pod \"dns-default-bdp7c\" (UID: \"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1\") " pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:59.014146 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.013949 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-trusted-ca\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.014146 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.013971 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-data-volume\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.014146 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.014012 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.014146 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.014041 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpnqk\" (UniqueName: \"kubernetes.io/projected/a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1-kube-api-access-xpnqk\") pod \"dns-default-bdp7c\" (UID: \"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1\") " pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:59.014703 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.014170 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6c5p\" (UniqueName: \"kubernetes.io/projected/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-kube-api-access-h6c5p\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.014703 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.014246 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.014703 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.014298 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1-tmp-dir\") pod \"dns-default-bdp7c\" (UID: \"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1\") " pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:59.014703 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.014327 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-image-registry-private-configuration\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.014703 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.014334 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-ca-trust-extracted\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.014703 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.014398 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-installation-pull-secrets\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.014703 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.014467 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-registry-certificates\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.014703 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.014491 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-bound-sa-token\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.014703 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.014637 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1-config-volume\") pod \"dns-default-bdp7c\" (UID: \"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1\") " pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:59.014703 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.014699 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1-tmp-dir\") pod \"dns-default-bdp7c\" (UID: \"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1\") " pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:59.015202 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.015108 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-trusted-ca\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.015339 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.015320 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-registry-certificates\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.018068 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.018045 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-image-registry-private-configuration\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.018068 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.018059 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1-metrics-tls\") pod \"dns-default-bdp7c\" (UID: \"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1\") " pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:59.018216 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.018150 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-installation-pull-secrets\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.018216 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.018177 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-registry-tls\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.026173 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.026140 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6c5p\" (UniqueName: \"kubernetes.io/projected/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-kube-api-access-h6c5p\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.026302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.026266 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5-bound-sa-token\") pod \"image-registry-66799f8b9-wfgqs\" (UID: \"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5\") " pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.026454 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.026435 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpnqk\" (UniqueName: \"kubernetes.io/projected/a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1-kube-api-access-xpnqk\") pod \"dns-default-bdp7c\" (UID: \"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1\") " pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:59.042380 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.042353 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:30:59.115447 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.115410 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-crio-socket\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.115447 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.115452 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bq7d\" (UniqueName: \"kubernetes.io/projected/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-kube-api-access-2bq7d\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.115680 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.115482 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-data-volume\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.115680 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.115498 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.115680 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.115523 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm2zx\" (UniqueName: \"kubernetes.io/projected/bf8a2f30-8ef2-4cb9-9d32-4beac035a680-kube-api-access-rm2zx\") pod \"ingress-canary-rh5rj\" (UID: \"bf8a2f30-8ef2-4cb9-9d32-4beac035a680\") " pod="openshift-ingress-canary/ingress-canary-rh5rj" Sep 06 10:30:59.115680 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.115547 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.115680 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.115611 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf8a2f30-8ef2-4cb9-9d32-4beac035a680-cert\") pod \"ingress-canary-rh5rj\" (UID: \"bf8a2f30-8ef2-4cb9-9d32-4beac035a680\") " pod="openshift-ingress-canary/ingress-canary-rh5rj" Sep 06 10:30:59.115680 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.115627 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-crio-socket\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.116003 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.115912 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-data-volume\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.116242 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.116223 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.117967 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.117949 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.123702 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.123676 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bq7d\" (UniqueName: \"kubernetes.io/projected/9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df-kube-api-access-2bq7d\") pod \"insights-runtime-extractor-56h9s\" (UID: \"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df\") " pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.125651 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.125635 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:30:59.139511 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.139485 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bdp7c" Sep 06 10:30:59.156523 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.156505 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-56h9s" Sep 06 10:30:59.216138 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.216094 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm2zx\" (UniqueName: \"kubernetes.io/projected/bf8a2f30-8ef2-4cb9-9d32-4beac035a680-kube-api-access-rm2zx\") pod \"ingress-canary-rh5rj\" (UID: \"bf8a2f30-8ef2-4cb9-9d32-4beac035a680\") " pod="openshift-ingress-canary/ingress-canary-rh5rj" Sep 06 10:30:59.216291 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.216200 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf8a2f30-8ef2-4cb9-9d32-4beac035a680-cert\") pod \"ingress-canary-rh5rj\" (UID: \"bf8a2f30-8ef2-4cb9-9d32-4beac035a680\") " pod="openshift-ingress-canary/ingress-canary-rh5rj" Sep 06 10:30:59.218714 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.218687 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf8a2f30-8ef2-4cb9-9d32-4beac035a680-cert\") pod \"ingress-canary-rh5rj\" (UID: \"bf8a2f30-8ef2-4cb9-9d32-4beac035a680\") " pod="openshift-ingress-canary/ingress-canary-rh5rj" Sep 06 10:30:59.224618 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.224588 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm2zx\" (UniqueName: \"kubernetes.io/projected/bf8a2f30-8ef2-4cb9-9d32-4beac035a680-kube-api-access-rm2zx\") pod \"ingress-canary-rh5rj\" (UID: \"bf8a2f30-8ef2-4cb9-9d32-4beac035a680\") " pod="openshift-ingress-canary/ingress-canary-rh5rj" Sep 06 10:30:59.244502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.244067 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rh5rj" Sep 06 10:30:59.459459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.459361 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bdp7c"] Sep 06 10:30:59.460705 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.460610 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66799f8b9-wfgqs"] Sep 06 10:30:59.462037 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.462016 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-56h9s"] Sep 06 10:30:59.463856 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:59.463831 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce928fb9_c6bb_4cfd_8e5c_e803f9990fb5.slice/crio-7e308de80f03936a04881ef70f25846cddc3eb8ac0e249a4b611cc6175bf10ce WatchSource:0}: Error finding container 7e308de80f03936a04881ef70f25846cddc3eb8ac0e249a4b611cc6175bf10ce: Status 404 returned error can't find the container with id 7e308de80f03936a04881ef70f25846cddc3eb8ac0e249a4b611cc6175bf10ce Sep 06 10:30:59.464058 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:59.464031 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a3fa7c_6e6f_4572_b490_c658b8ff6bc1.slice/crio-7e5d1f709755e03d3c29f48207025a33c0c7ee6e45b54bac4e748d1677af6ebb WatchSource:0}: Error finding container 7e5d1f709755e03d3c29f48207025a33c0c7ee6e45b54bac4e748d1677af6ebb: Status 404 returned error can't find the container with id 7e5d1f709755e03d3c29f48207025a33c0c7ee6e45b54bac4e748d1677af6ebb Sep 06 10:30:59.539918 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.539892 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rh5rj"] Sep 06 10:30:59.548176 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:30:59.548149 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf8a2f30_8ef2_4cb9_9d32_4beac035a680.slice/crio-3283eef7686a6f9e9802f885481a97065170707101e1a7823edb5cee47f7d5cc WatchSource:0}: Error finding container 3283eef7686a6f9e9802f885481a97065170707101e1a7823edb5cee47f7d5cc: Status 404 returned error can't find the container with id 3283eef7686a6f9e9802f885481a97065170707101e1a7823edb5cee47f7d5cc Sep 06 10:30:59.920906 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.920871 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:30:59.920906 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.920894 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:30:59.923659 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.923636 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 06 10:30:59.923756 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.923701 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 06 10:30:59.923910 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.923895 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h4kvp\"" Sep 06 10:30:59.924301 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:30:59.924273 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 06 10:31:00.142154 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.142064 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rh5rj" event={"ID":"bf8a2f30-8ef2-4cb9-9d32-4beac035a680","Type":"ContainerStarted","Data":"3283eef7686a6f9e9802f885481a97065170707101e1a7823edb5cee47f7d5cc"} Sep 06 10:31:00.144967 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.144455 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" event={"ID":"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5","Type":"ContainerStarted","Data":"2b9aa9df8bf74a85415fa7d1cf40facdcd3f3d3a93270b5c3f37ffaab03ea68e"} Sep 06 10:31:00.144967 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.144508 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" event={"ID":"ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5","Type":"ContainerStarted","Data":"7e308de80f03936a04881ef70f25846cddc3eb8ac0e249a4b611cc6175bf10ce"} Sep 06 10:31:00.144967 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.144568 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:31:00.146627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.146530 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-56h9s" event={"ID":"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df","Type":"ContainerStarted","Data":"dd7807432eb5217081cc762c4b2eae4d9643acd67240bd7f01956ab53fa774d2"} Sep 06 10:31:00.146627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.146582 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-56h9s" event={"ID":"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df","Type":"ContainerStarted","Data":"a05a2c34e906466ee01311cc3d7cb5fe0ef305ae2d314455dc5d40c46834d4c7"} Sep 06 10:31:00.148390 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.148364 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bdp7c" event={"ID":"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1","Type":"ContainerStarted","Data":"7e5d1f709755e03d3c29f48207025a33c0c7ee6e45b54bac4e748d1677af6ebb"} Sep 06 10:31:00.162194 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.162027 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" podStartSLOduration=3.162005688 podStartE2EDuration="3.162005688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:00.161483717 +0000 UTC m=+53.743973364" watchObservedRunningTime="2026-09-06 10:31:00.162005688 +0000 UTC m=+53.744495317" Sep 06 10:31:00.715397 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.715360 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:00.803717 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.803680 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:00.844117 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.844079 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx"] Sep 06 10:31:00.846069 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.846049 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx" Sep 06 10:31:00.848257 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.848236 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 06 10:31:00.848363 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.848313 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-g86hw\"" Sep 06 10:31:00.855608 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.855535 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx"] Sep 06 10:31:00.879283 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:00.879247 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:01.013412 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:01.013337 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:01.032135 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:01.032090 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/295fe1ab-b385-4ed7-b35f-f8cc4b9037b5-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-nsdkx\" (UID: \"295fe1ab-b385-4ed7-b35f-f8cc4b9037b5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx" Sep 06 10:31:01.133370 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:01.133340 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/295fe1ab-b385-4ed7-b35f-f8cc4b9037b5-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-nsdkx\" (UID: \"295fe1ab-b385-4ed7-b35f-f8cc4b9037b5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx" Sep 06 10:31:01.135997 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:01.135958 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/295fe1ab-b385-4ed7-b35f-f8cc4b9037b5-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-nsdkx\" (UID: \"295fe1ab-b385-4ed7-b35f-f8cc4b9037b5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx" Sep 06 10:31:01.153510 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:01.153477 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-56h9s" event={"ID":"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df","Type":"ContainerStarted","Data":"1888b19f2055d76d68c8b6eca85a914fa395a7ca48fa6cdb55d859cde278b730"} Sep 06 10:31:01.157608 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:01.157325 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bdp7c" event={"ID":"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1","Type":"ContainerStarted","Data":"60606335200bab2c5a739c385e18221a17c3b57977fca6661f17edab9c24885e"} Sep 06 10:31:01.157608 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:01.157436 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx" Sep 06 10:31:01.446029 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:01.445997 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx"] Sep 06 10:31:02.005070 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:31:02.005035 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod295fe1ab_b385_4ed7_b35f_f8cc4b9037b5.slice/crio-529c42d34652ae357c6b2f59b161184e59f80165ccdb07d277fda39b0c397a8c WatchSource:0}: Error finding container 529c42d34652ae357c6b2f59b161184e59f80165ccdb07d277fda39b0c397a8c: Status 404 returned error can't find the container with id 529c42d34652ae357c6b2f59b161184e59f80165ccdb07d277fda39b0c397a8c Sep 06 10:31:02.161018 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:02.160976 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx" event={"ID":"295fe1ab-b385-4ed7-b35f-f8cc4b9037b5","Type":"ContainerStarted","Data":"529c42d34652ae357c6b2f59b161184e59f80165ccdb07d277fda39b0c397a8c"} Sep 06 10:31:02.163171 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:02.163141 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bdp7c" event={"ID":"a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1","Type":"ContainerStarted","Data":"d6c3bb5f4cb655220e38d9198088edf847c471b9727a2698671e81b43f62984f"} Sep 06 10:31:02.163313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:02.163270 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bdp7c" Sep 06 10:31:02.178781 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:02.178723 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bdp7c" podStartSLOduration=2.6081446550000003 podStartE2EDuration="4.178702736s" totalImagesPullingTime="1.570558081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:02.178495332 +0000 UTC m=+55.760984963" watchObservedRunningTime="2026-09-06 10:31:02.178702736 +0000 UTC m=+55.761192368" Sep 06 10:31:02.682746 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:02.682707 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:02.737410 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:02.737371 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:03.168177 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:03.168144 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rh5rj" event={"ID":"bf8a2f30-8ef2-4cb9-9d32-4beac035a680","Type":"ContainerStarted","Data":"a360284998473c85f289dcd3dc4fdf867c9fcc230a5fb3f5a61ab19a0cf4d671"} Sep 06 10:31:03.170462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:03.170437 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-56h9s" event={"ID":"9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df","Type":"ContainerStarted","Data":"7d156c3f6a32660350c4e8fd3e1b9432001550ec17400327595658a18e3763e2"} Sep 06 10:31:03.172006 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:03.171981 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx" event={"ID":"295fe1ab-b385-4ed7-b35f-f8cc4b9037b5","Type":"ContainerStarted","Data":"8f806c838b65090d702a698bb93963dec55512fc6e4a75ae641ed34505cd8bda"} Sep 06 10:31:03.172267 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:03.172249 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx" Sep 06 10:31:03.176632 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:03.176612 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx" Sep 06 10:31:03.183053 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:03.183013 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rh5rj" podStartSLOduration=2.334271637 podStartE2EDuration="5.183001899s" totalImagesPullingTime="2.848730262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:03.18289484 +0000 UTC m=+56.765384471" watchObservedRunningTime="2026-09-06 10:31:03.183001899 +0000 UTC m=+56.765491587" Sep 06 10:31:03.214751 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:03.214699 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-nsdkx" podStartSLOduration=2.155689248 podStartE2EDuration="3.214681036s" totalImagesPullingTime="1.058991788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:03.197532318 +0000 UTC m=+56.780021949" watchObservedRunningTime="2026-09-06 10:31:03.214681036 +0000 UTC m=+56.797170667" Sep 06 10:31:03.841888 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:03.841852 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:03.917906 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:03.917873 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:03.977696 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:03.977659 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:06.086106 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:06.086077 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cj6jp" Sep 06 10:31:06.111662 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:06.111609 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-56h9s" podStartSLOduration=5.372673641 podStartE2EDuration="8.111590012s" totalImagesPullingTime="2.738916371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:03.214216174 +0000 UTC m=+56.796705806" watchObservedRunningTime="2026-09-06 10:31:06.111590012 +0000 UTC m=+59.694079643" Sep 06 10:31:06.238172 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:06.238140 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:06.338985 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:06.338871 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:07.021429 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:07.021400 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:07.082505 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:07.082470 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:07.138375 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:07.138345 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:07.189059 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:07.189030 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:07.488022 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:07.487991 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:07.589563 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:07.589529 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:07.642286 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:07.642254 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:08.331809 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:08.331780 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:08.432141 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:08.432084 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:08.529931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:08.529901 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:08.794745 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:08.794711 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:08.859902 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:08.859866 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:10.024609 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.024543 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:10.232453 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.232423 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:10.331485 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.331401 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr"] Sep 06 10:31:10.334523 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.334500 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:10.340895 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.340850 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:10.341982 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.341962 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 06 10:31:10.342085 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.342026 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 06 10:31:10.342983 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.342956 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-jp4bh\"" Sep 06 10:31:10.343056 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.342959 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 06 10:31:10.343056 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.342959 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 06 10:31:10.343056 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.342957 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 06 10:31:10.355923 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.355900 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr"] Sep 06 10:31:10.385155 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.385110 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-skwvr"] Sep 06 10:31:10.387233 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.387211 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.390451 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.390430 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 06 10:31:10.390573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.390525 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v6mwz\"" Sep 06 10:31:10.391047 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.391029 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 06 10:31:10.391512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.391496 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 06 10:31:10.400653 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400634 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a560b82a-e82b-487d-bd6c-a27417548ae4-sys\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.400737 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400666 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4wwq\" (UniqueName: \"kubernetes.io/projected/691ff084-48cd-4dce-ad4a-4cc02968306f-kube-api-access-q4wwq\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:10.400737 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400684 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.400737 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400701 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-accelerators-collector-config\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.400737 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400719 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/691ff084-48cd-4dce-ad4a-4cc02968306f-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:10.400863 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a560b82a-e82b-487d-bd6c-a27417548ae4-root\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.400863 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400805 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-textfile\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.400863 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400838 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/691ff084-48cd-4dce-ad4a-4cc02968306f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:10.400949 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400888 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/691ff084-48cd-4dce-ad4a-4cc02968306f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:10.400949 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400909 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-tls\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.400949 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400923 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4ltv\" (UniqueName: \"kubernetes.io/projected/a560b82a-e82b-487d-bd6c-a27417548ae4-kube-api-access-d4ltv\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.401039 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400952 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-wtmp\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.401039 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.400978 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a560b82a-e82b-487d-bd6c-a27417548ae4-metrics-client-ca\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.456720 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.456691 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:10.501882 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.501829 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502023 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.501889 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-accelerators-collector-config\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502023 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.501926 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/691ff084-48cd-4dce-ad4a-4cc02968306f-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:10.502023 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.501957 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a560b82a-e82b-487d-bd6c-a27417548ae4-root\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502023 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.501986 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-textfile\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502023 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502009 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/691ff084-48cd-4dce-ad4a-4cc02968306f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:10.502302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502084 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a560b82a-e82b-487d-bd6c-a27417548ae4-root\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502302 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:31:10.502115 2574 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 06 10:31:10.502302 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:31:10.502210 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/691ff084-48cd-4dce-ad4a-4cc02968306f-openshift-state-metrics-tls podName:691ff084-48cd-4dce-ad4a-4cc02968306f nodeName:}" failed. No retries permitted until 2026-09-06 10:31:11.002190973 +0000 UTC m=+64.584680583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/691ff084-48cd-4dce-ad4a-4cc02968306f-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-hv5vr" (UID: "691ff084-48cd-4dce-ad4a-4cc02968306f") : secret "openshift-state-metrics-tls" not found Sep 06 10:31:10.502302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502274 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/691ff084-48cd-4dce-ad4a-4cc02968306f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:10.502502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502326 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-tls\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4ltv\" (UniqueName: \"kubernetes.io/projected/a560b82a-e82b-487d-bd6c-a27417548ae4-kube-api-access-d4ltv\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502402 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-wtmp\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502425 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a560b82a-e82b-487d-bd6c-a27417548ae4-metrics-client-ca\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502502 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:31:10.502446 2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 06 10:31:10.502502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502467 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a560b82a-e82b-487d-bd6c-a27417548ae4-sys\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502483 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-textfile\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502820 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:31:10.502511 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-tls podName:a560b82a-e82b-487d-bd6c-a27417548ae4 nodeName:}" failed. No retries permitted until 2026-09-06 10:31:11.002482227 +0000 UTC m=+64.584971859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-tls") pod "node-exporter-skwvr" (UID: "a560b82a-e82b-487d-bd6c-a27417548ae4") : secret "node-exporter-tls" not found Sep 06 10:31:10.502820 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502518 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a560b82a-e82b-487d-bd6c-a27417548ae4-sys\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502820 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502589 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4wwq\" (UniqueName: \"kubernetes.io/projected/691ff084-48cd-4dce-ad4a-4cc02968306f-kube-api-access-q4wwq\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:10.502820 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502625 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-wtmp\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502820 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502723 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-accelerators-collector-config\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.502992 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502815 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/691ff084-48cd-4dce-ad4a-4cc02968306f-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:10.502992 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.502938 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a560b82a-e82b-487d-bd6c-a27417548ae4-metrics-client-ca\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.504595 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.504564 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.504726 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.504709 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/691ff084-48cd-4dce-ad4a-4cc02968306f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:10.514399 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.514374 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4wwq\" (UniqueName: \"kubernetes.io/projected/691ff084-48cd-4dce-ad4a-4cc02968306f-kube-api-access-q4wwq\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:10.514517 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.514501 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4ltv\" (UniqueName: \"kubernetes.io/projected/a560b82a-e82b-487d-bd6c-a27417548ae4-kube-api-access-d4ltv\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:10.560310 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.560277 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:10.657978 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.657897 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:10.758361 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.758330 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:10.886163 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:10.886094 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:11.005617 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.005573 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/691ff084-48cd-4dce-ad4a-4cc02968306f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:11.005800 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.005639 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-tls\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:11.008036 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.008001 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a560b82a-e82b-487d-bd6c-a27417548ae4-node-exporter-tls\") pod \"node-exporter-skwvr\" (UID: \"a560b82a-e82b-487d-bd6c-a27417548ae4\") " pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:11.008167 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.008058 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/691ff084-48cd-4dce-ad4a-4cc02968306f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-hv5vr\" (UID: \"691ff084-48cd-4dce-ad4a-4cc02968306f\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:11.243451 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.243410 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" Sep 06 10:31:11.248036 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.247986 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:11.296523 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.296454 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-skwvr" Sep 06 10:31:11.331798 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.331769 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:11.337294 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.337270 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 06 10:31:11.340206 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.340191 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.342574 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.342553 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 06 10:31:11.342694 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.342602 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x6xvm\"" Sep 06 10:31:11.342694 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.342609 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 06 10:31:11.342694 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.342612 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 06 10:31:11.342694 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.342667 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 06 10:31:11.343034 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.343010 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 06 10:31:11.343034 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.343028 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 06 10:31:11.343156 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.343099 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 06 10:31:11.343216 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.343183 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 06 10:31:11.343216 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.343211 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 06 10:31:11.355009 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.354984 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 06 10:31:11.408359 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408324 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-config-volume\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.408359 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408356 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/db4211e5-0dd2-48bd-a4b8-f891dd452348-config-out\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.408359 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408371 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.408662 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408391 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.408662 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408411 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/db4211e5-0dd2-48bd-a4b8-f891dd452348-tls-assets\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.408662 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408469 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.408662 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408577 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.408662 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408616 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.408662 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408645 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dxth\" (UniqueName: \"kubernetes.io/projected/db4211e5-0dd2-48bd-a4b8-f891dd452348-kube-api-access-6dxth\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.408874 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408685 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-web-config\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.408874 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408751 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.408874 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408771 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.408874 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.408789 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509223 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509190 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-web-config\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509352 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509262 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509352 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509288 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509352 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509312 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509352 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509343 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-config-volume\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509362 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/db4211e5-0dd2-48bd-a4b8-f891dd452348-config-out\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509382 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509417 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509447 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/db4211e5-0dd2-48bd-a4b8-f891dd452348-tls-assets\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509470 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509523 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509551 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509576 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dxth\" (UniqueName: \"kubernetes.io/projected/db4211e5-0dd2-48bd-a4b8-f891dd452348-kube-api-access-6dxth\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.509941 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.509870 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.510067 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:31:11.509980 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-trusted-ca-bundle podName:db4211e5-0dd2-48bd-a4b8-f891dd452348 nodeName:}" failed. No retries permitted until 2026-09-06 10:31:12.009956516 +0000 UTC m=+65.592446126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348") : configmap references non-existent config key: ca-bundle.crt Sep 06 10:31:11.510246 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:31:11.510225 2574 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 06 10:31:11.510339 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:31:11.510304 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-main-tls podName:db4211e5-0dd2-48bd-a4b8-f891dd452348 nodeName:}" failed. No retries permitted until 2026-09-06 10:31:12.01028378 +0000 UTC m=+65.592773402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348") : secret "alertmanager-main-tls" not found Sep 06 10:31:11.510846 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.510819 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.512690 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.512329 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-web-config\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.512690 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.512389 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.512690 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.512600 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr"] Sep 06 10:31:11.512690 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.512645 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-config-volume\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.513867 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.513354 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.513867 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.513819 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/db4211e5-0dd2-48bd-a4b8-f891dd452348-config-out\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.514099 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.514078 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.514666 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.514642 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/db4211e5-0dd2-48bd-a4b8-f891dd452348-tls-assets\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.514754 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.514662 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.517680 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:31:11.517657 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod691ff084_48cd_4dce_ad4a_4cc02968306f.slice/crio-07723ba43e2611c813009f40872e3cba3fb8415738e062d32e41b002a81ff49e WatchSource:0}: Error finding container 07723ba43e2611c813009f40872e3cba3fb8415738e062d32e41b002a81ff49e: Status 404 returned error can't find the container with id 07723ba43e2611c813009f40872e3cba3fb8415738e062d32e41b002a81ff49e Sep 06 10:31:11.520886 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.520868 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dxth\" (UniqueName: \"kubernetes.io/projected/db4211e5-0dd2-48bd-a4b8-f891dd452348-kube-api-access-6dxth\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:11.597060 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.597019 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:11.926481 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:11.926450 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:12.012534 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.012501 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:12.012534 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.012545 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:12.013422 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.013389 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:12.015302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.015275 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:12.174671 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.174645 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bdp7c" Sep 06 10:31:12.202209 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.202175 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-skwvr" event={"ID":"a560b82a-e82b-487d-bd6c-a27417548ae4","Type":"ContainerStarted","Data":"ee17a9227305680fdf759b5ecb8159c2e51598b8e5357a6c8276970c9c197b7b"} Sep 06 10:31:12.204355 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.204332 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" event={"ID":"691ff084-48cd-4dce-ad4a-4cc02968306f","Type":"ContainerStarted","Data":"1fa7e4ac81aac8719aaa1dfd3d234d07c42373f9dc9dbf08a7705563d424f903"} Sep 06 10:31:12.204462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.204381 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" event={"ID":"691ff084-48cd-4dce-ad4a-4cc02968306f","Type":"ContainerStarted","Data":"18bb3bd373c4d1783c5cfe4f101810c55596f09b768563dcb16dba6406c5713d"} Sep 06 10:31:12.204462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.204400 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" event={"ID":"691ff084-48cd-4dce-ad4a-4cc02968306f","Type":"ContainerStarted","Data":"07723ba43e2611c813009f40872e3cba3fb8415738e062d32e41b002a81ff49e"} Sep 06 10:31:12.249591 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.249565 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:31:12.357647 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.357618 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7f855d5774-qgpvp"] Sep 06 10:31:12.383328 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.383295 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f855d5774-qgpvp"] Sep 06 10:31:12.383503 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.383426 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.385891 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.385863 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 06 10:31:12.386520 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.386152 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 06 10:31:12.386520 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.386246 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 06 10:31:12.386520 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.386249 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-gp72c\"" Sep 06 10:31:12.386824 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.386803 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6g1f9ia5sfoif\"" Sep 06 10:31:12.387069 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.387053 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 06 10:31:12.387183 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.387151 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 06 10:31:12.415039 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.415016 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxkx8\" (UniqueName: \"kubernetes.io/projected/f21b4bb0-7741-421a-a77c-95b7c59a79a0-kube-api-access-zxkx8\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.415159 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.415045 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f21b4bb0-7741-421a-a77c-95b7c59a79a0-metrics-client-ca\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.415221 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.415167 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-grpc-tls\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.415221 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.415204 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-tls\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.415323 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.415239 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.415323 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.415271 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.415430 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.415322 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.415430 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.415395 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.516712 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.516641 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-grpc-tls\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.516712 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.516688 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-tls\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.516908 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.516717 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.516908 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.516748 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.516908 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.516795 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.516908 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.516849 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.517104 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.516979 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxkx8\" (UniqueName: \"kubernetes.io/projected/f21b4bb0-7741-421a-a77c-95b7c59a79a0-kube-api-access-zxkx8\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.517104 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.517035 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f21b4bb0-7741-421a-a77c-95b7c59a79a0-metrics-client-ca\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.517816 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.517787 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f21b4bb0-7741-421a-a77c-95b7c59a79a0-metrics-client-ca\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.519728 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.519703 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-grpc-tls\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.520170 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.520103 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.520270 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.520221 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.520418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.520395 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.520763 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.520724 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-tls\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.521270 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.521231 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f21b4bb0-7741-421a-a77c-95b7c59a79a0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.524622 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.524603 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxkx8\" (UniqueName: \"kubernetes.io/projected/f21b4bb0-7741-421a-a77c-95b7c59a79a0-kube-api-access-zxkx8\") pod \"thanos-querier-7f855d5774-qgpvp\" (UID: \"f21b4bb0-7741-421a-a77c-95b7c59a79a0\") " pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.540914 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.540894 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 06 10:31:12.543708 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:31:12.543678 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb4211e5_0dd2_48bd_a4b8_f891dd452348.slice/crio-4653324f363b2a1f07cec33f7fbb0ba033885fee52eb385efad31836ea216363 WatchSource:0}: Error finding container 4653324f363b2a1f07cec33f7fbb0ba033885fee52eb385efad31836ea216363: Status 404 returned error can't find the container with id 4653324f363b2a1f07cec33f7fbb0ba033885fee52eb385efad31836ea216363 Sep 06 10:31:12.618062 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.618031 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:31:12.620605 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.620580 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 06 10:31:12.630592 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.630571 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d037fa14-5fbf-4635-92df-3b0117254fca-original-pull-secret\") pod \"global-pull-secret-syncer-n424n\" (UID: \"d037fa14-5fbf-4635-92df-3b0117254fca\") " pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:31:12.693785 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.693743 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:12.718927 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.718904 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:31:12.721262 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.721238 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 06 10:31:12.732460 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.732431 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23d754da-9be5-4fe4-9d31-0ce5f1c3b63e-metrics-certs\") pod \"network-metrics-daemon-tg7pf\" (UID: \"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e\") " pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:31:12.751118 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.751096 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bx9l2\"" Sep 06 10:31:12.760087 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.760061 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tg7pf" Sep 06 10:31:12.820237 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.820204 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpfh8\" (UniqueName: \"kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8\") pod \"network-check-target-lrpg9\" (UID: \"dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4\") " pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:31:12.822362 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.822345 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 06 10:31:12.833620 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.833592 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 06 10:31:12.838618 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.838604 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n424n" Sep 06 10:31:12.843858 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:12.843836 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpfh8\" (UniqueName: \"kubernetes.io/projected/dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4-kube-api-access-dpfh8\") pod \"network-check-target-lrpg9\" (UID: \"dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4\") " pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:31:13.135419 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.135344 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h4kvp\"" Sep 06 10:31:13.144076 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.144051 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:31:13.158212 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.158180 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n424n"] Sep 06 10:31:13.163569 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.163550 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tg7pf"] Sep 06 10:31:13.169472 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.169451 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f855d5774-qgpvp"] Sep 06 10:31:13.209183 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.209154 2574 generic.go:356] "Generic (PLEG): container finished" podID="a560b82a-e82b-487d-bd6c-a27417548ae4" containerID="0fdf7727a3a0200fb5696fbfa2099777ae6bdcbacf9a4a6fe4fd949103114a48" exitCode=0 Sep 06 10:31:13.209335 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.209241 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-skwvr" event={"ID":"a560b82a-e82b-487d-bd6c-a27417548ae4","Type":"ContainerDied","Data":"0fdf7727a3a0200fb5696fbfa2099777ae6bdcbacf9a4a6fe4fd949103114a48"} Sep 06 10:31:13.210429 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.210409 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerStarted","Data":"4653324f363b2a1f07cec33f7fbb0ba033885fee52eb385efad31836ea216363"} Sep 06 10:31:13.214000 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.213977 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:13.229572 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:31:13.229543 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd037fa14_5fbf_4635_92df_3b0117254fca.slice/crio-0a844b1afef7c74fc89c96e4e57df98ed9ef7b1cadc90e471b6845cdaab3b9de WatchSource:0}: Error finding container 0a844b1afef7c74fc89c96e4e57df98ed9ef7b1cadc90e471b6845cdaab3b9de: Status 404 returned error can't find the container with id 0a844b1afef7c74fc89c96e4e57df98ed9ef7b1cadc90e471b6845cdaab3b9de Sep 06 10:31:13.230011 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:31:13.229992 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23d754da_9be5_4fe4_9d31_0ce5f1c3b63e.slice/crio-81cdc762bac805b0e1315c1dfe1a0d78febb77655367a92158923edd66549848 WatchSource:0}: Error finding container 81cdc762bac805b0e1315c1dfe1a0d78febb77655367a92158923edd66549848: Status 404 returned error can't find the container with id 81cdc762bac805b0e1315c1dfe1a0d78febb77655367a92158923edd66549848 Sep 06 10:31:13.231102 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:31:13.231023 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf21b4bb0_7741_421a_a77c_95b7c59a79a0.slice/crio-07c14403784e0634dee93e9935469d93605db34f427cd947e189893a2beda812 WatchSource:0}: Error finding container 07c14403784e0634dee93e9935469d93605db34f427cd947e189893a2beda812: Status 404 returned error can't find the container with id 07c14403784e0634dee93e9935469d93605db34f427cd947e189893a2beda812 Sep 06 10:31:13.313859 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.313827 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:13.413620 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.413591 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:13.523743 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.523658 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lrpg9"] Sep 06 10:31:13.528577 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:31:13.528541 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf4cc4f_9daf_490e_ab6b_6529c3cf9fd4.slice/crio-d76b72fb34611bf814d2647d8c1fa41cddcd609d912d49d280fd52a714e74b75 WatchSource:0}: Error finding container d76b72fb34611bf814d2647d8c1fa41cddcd609d912d49d280fd52a714e74b75: Status 404 returned error can't find the container with id d76b72fb34611bf814d2647d8c1fa41cddcd609d912d49d280fd52a714e74b75 Sep 06 10:31:13.769532 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:13.769493 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:14.219164 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:14.219081 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" event={"ID":"691ff084-48cd-4dce-ad4a-4cc02968306f","Type":"ContainerStarted","Data":"be4a0b658143eac8ba889296b402709e6bdb0149591736e2085b6e6feb6c352b"} Sep 06 10:31:14.220819 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:14.220783 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lrpg9" event={"ID":"dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4","Type":"ContainerStarted","Data":"d76b72fb34611bf814d2647d8c1fa41cddcd609d912d49d280fd52a714e74b75"} Sep 06 10:31:14.222054 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:14.222008 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n424n" event={"ID":"d037fa14-5fbf-4635-92df-3b0117254fca","Type":"ContainerStarted","Data":"0a844b1afef7c74fc89c96e4e57df98ed9ef7b1cadc90e471b6845cdaab3b9de"} Sep 06 10:31:14.225935 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:14.225910 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-skwvr" event={"ID":"a560b82a-e82b-487d-bd6c-a27417548ae4","Type":"ContainerStarted","Data":"176c69a2675efd05ee63bc3298a4d4bbaf6b4a8e7c538c06e69ed786d3aa9b64"} Sep 06 10:31:14.226038 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:14.225957 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-skwvr" event={"ID":"a560b82a-e82b-487d-bd6c-a27417548ae4","Type":"ContainerStarted","Data":"dab5a0553fb1d4fb1bbad6ba9345257e069b856c822fb42abff0784e29ebc63f"} Sep 06 10:31:14.228314 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:14.228288 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" event={"ID":"f21b4bb0-7741-421a-a77c-95b7c59a79a0","Type":"ContainerStarted","Data":"07c14403784e0634dee93e9935469d93605db34f427cd947e189893a2beda812"} Sep 06 10:31:14.229827 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:14.229801 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tg7pf" event={"ID":"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e","Type":"ContainerStarted","Data":"81cdc762bac805b0e1315c1dfe1a0d78febb77655367a92158923edd66549848"} Sep 06 10:31:14.258305 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:14.258253 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-skwvr" podStartSLOduration=3.363274335 podStartE2EDuration="4.258235859s" totalImagesPullingTime="819.773393ms" totalInitContainerRuntime="75.188131ms" isStatefulPod=true podCreationTimestamp="2026-09-06 10:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:14.256453493 +0000 UTC m=+67.838943139" watchObservedRunningTime="2026-09-06 10:31:14.258235859 +0000 UTC m=+67.840725491" Sep 06 10:31:14.258640 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:14.258604 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-hv5vr" podStartSLOduration=2.639128587 podStartE2EDuration="4.258590626s" totalImagesPullingTime="1.619462039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:14.239416755 +0000 UTC m=+67.821906387" watchObservedRunningTime="2026-09-06 10:31:14.258590626 +0000 UTC m=+67.841080258" Sep 06 10:31:15.028610 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:15.028556 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:15.238039 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:15.238001 2574 generic.go:356] "Generic (PLEG): container finished" podID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerID="0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c" exitCode=0 Sep 06 10:31:15.238243 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:15.238116 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerDied","Data":"0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c"} Sep 06 10:31:15.241637 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:15.241588 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tg7pf" event={"ID":"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e","Type":"ContainerStarted","Data":"72e82daaec3b84d7216793e84bed4735a874a02847cab8ee2aec7e6a07275cf6"} Sep 06 10:31:15.241726 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:15.241643 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tg7pf" event={"ID":"23d754da-9be5-4fe4-9d31-0ce5f1c3b63e","Type":"ContainerStarted","Data":"df9690a4d08ef23c325abb79f37d28dfb7f7d90288dc6733394fd6e2af2c5873"} Sep 06 10:31:15.287512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:15.287331 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tg7pf" podStartSLOduration=67.026854076 podStartE2EDuration="1m8.287308144s" totalImagesPullingTime="1.260454068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:15.286145766 +0000 UTC m=+68.868635394" watchObservedRunningTime="2026-09-06 10:31:15.287308144 +0000 UTC m=+68.869797777" Sep 06 10:31:15.404516 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:15.404461 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:15.501880 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:15.501845 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:15.565663 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:15.565530 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:16.018909 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:16.018833 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:16.113692 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:16.113600 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:16.216132 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:16.216083 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:16.250328 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:16.250290 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" event={"ID":"f21b4bb0-7741-421a-a77c-95b7c59a79a0","Type":"ContainerStarted","Data":"d814aa7ca4435c14cbaa6445a4b6e206e21e5de0e07749e1acc8cd1912370226"} Sep 06 10:31:16.250532 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:16.250371 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" event={"ID":"f21b4bb0-7741-421a-a77c-95b7c59a79a0","Type":"ContainerStarted","Data":"08fd5655bc6340089044d19883201d4f107bdc49d3c706a3118be007ff7e1e76"} Sep 06 10:31:16.250532 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:16.250418 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" event={"ID":"f21b4bb0-7741-421a-a77c-95b7c59a79a0","Type":"ContainerStarted","Data":"33278b84b58ae999edfcabf0f87b9aa7bc885eb2147f9f859a65d07b3af502cc"} Sep 06 10:31:16.314033 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:16.313942 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:16.497141 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:16.497086 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:16.696439 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:16.696404 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:16.791069 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:16.790987 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:17.083428 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:17.083328 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:17.184476 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:17.184440 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:17.243053 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:17.243022 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:17.306556 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:17.306515 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:17.357416 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:17.357335 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:17.697203 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:17.697157 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:17.889081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:17.889047 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:17.941797 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:17.941754 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:18.680178 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:18.680145 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:18.779185 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:18.779153 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:18.841084 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:18.841043 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:19.131564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:19.131494 2574 patch_prober.go:28] interesting pod/image-registry-66799f8b9-wfgqs container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 06 10:31:19.131735 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:19.131554 2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" podUID="ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 06 10:31:19.270047 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:19.270012 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerStarted","Data":"73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971"} Sep 06 10:31:19.270183 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:19.270101 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerStarted","Data":"8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37"} Sep 06 10:31:19.270183 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:19.270170 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerStarted","Data":"88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704"} Sep 06 10:31:19.272434 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:19.272396 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lrpg9" event={"ID":"dcf4cc4f-9daf-490e-ab6b-6529c3cf9fd4","Type":"ContainerStarted","Data":"28ce6491b08fd1b7cb1c1df8280374645b8de1e372dbe2bc9a5ba66b22dbbd45"} Sep 06 10:31:19.272619 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:19.272460 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:31:19.274082 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:19.274058 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n424n" event={"ID":"d037fa14-5fbf-4635-92df-3b0117254fca","Type":"ContainerStarted","Data":"399d9557d063a9b917e6e672516d36677cdf9e24dd8790367a05db33e113a3bf"} Sep 06 10:31:19.291675 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:19.291463 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lrpg9" podStartSLOduration=66.824205508 podStartE2EDuration="1m12.291442983s" totalImagesPullingTime="5.467237475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:19.289996313 +0000 UTC m=+72.872485946" watchObservedRunningTime="2026-09-06 10:31:19.291442983 +0000 UTC m=+72.873932617" Sep 06 10:31:19.307926 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:19.307874 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n424n" podStartSLOduration=66.576896681 podStartE2EDuration="1m12.307853117s" totalImagesPullingTime="5.730956436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 10:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:19.307008057 +0000 UTC m=+72.889497691" watchObservedRunningTime="2026-09-06 10:31:19.307853117 +0000 UTC m=+72.890342749" Sep 06 10:31:19.317203 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:19.317179 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:19.375360 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:19.375330 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:20.281370 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:20.281333 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerStarted","Data":"daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09"} Sep 06 10:31:20.281785 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:20.281402 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerStarted","Data":"103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300"} Sep 06 10:31:20.281785 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:20.281438 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerStarted","Data":"85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065"} Sep 06 10:31:20.284110 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:20.284083 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" event={"ID":"f21b4bb0-7741-421a-a77c-95b7c59a79a0","Type":"ContainerStarted","Data":"ea1adaea6a6f884979e0d8f2a2956abbd3a40e47b2fc2d8ecc57ab7b72586fa5"} Sep 06 10:31:20.284245 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:20.284159 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" event={"ID":"f21b4bb0-7741-421a-a77c-95b7c59a79a0","Type":"ContainerStarted","Data":"a20ea19dd109a665d909b1baec18e3cc6479f1fc01851d95e43d1a4c56ca33f3"} Sep 06 10:31:20.284245 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:20.284193 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" event={"ID":"f21b4bb0-7741-421a-a77c-95b7c59a79a0","Type":"ContainerStarted","Data":"5c0d643c839613525e0a99d3901720b4894856a0d478b495318f575b48b1225b"} Sep 06 10:31:20.311661 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:20.311614 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.581688922 podStartE2EDuration="9.3115995s" totalImagesPullingTime="5.706864871s" totalInitContainerRuntime="23.045707ms" isStatefulPod=false podCreationTimestamp="2026-09-06 10:31:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:20.309154751 +0000 UTC m=+73.891644382" watchObservedRunningTime="2026-09-06 10:31:20.3115995 +0000 UTC m=+73.894089190" Sep 06 10:31:20.337346 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:20.337289 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" podStartSLOduration=2.359300771 podStartE2EDuration="8.3372696s" totalImagesPullingTime="5.977968829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:31:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:20.336393236 +0000 UTC m=+73.918882866" watchObservedRunningTime="2026-09-06 10:31:20.3372696 +0000 UTC m=+73.919759224" Sep 06 10:31:21.161714 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:21.161685 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66799f8b9-wfgqs" Sep 06 10:31:21.289049 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:21.289018 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:21.294069 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:21.294046 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7f855d5774-qgpvp" Sep 06 10:31:21.299187 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:21.299165 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:21.394515 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:21.394482 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:21.495176 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:21.495116 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:23.741955 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:23.741927 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:23.797147 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:23.797105 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:25.417139 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:25.417099 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:25.487236 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:25.487199 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:25.851623 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:25.851539 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:25.952505 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:25.952477 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:26.714921 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:26.714889 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:26.815979 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:26.815953 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:27.144586 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.144513 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:27.248031 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.248000 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:27.441010 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.440978 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cfd87f5b5-47mdc"] Sep 06 10:31:27.445082 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.445067 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.447597 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.447576 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 06 10:31:27.447722 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.447628 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 06 10:31:27.448737 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.448716 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 06 10:31:27.448737 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.448735 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 06 10:31:27.448909 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.448749 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 06 10:31:27.448909 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.448789 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9qlhj\"" Sep 06 10:31:27.448909 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.448732 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 06 10:31:27.448909 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.448867 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 06 10:31:27.454552 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.454532 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cfd87f5b5-47mdc"] Sep 06 10:31:27.546032 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.545999 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-config\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.546211 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.546044 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-service-ca\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.546211 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.546074 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l5pp\" (UniqueName: \"kubernetes.io/projected/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-kube-api-access-2l5pp\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.546211 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.546116 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-oauth-config\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.546211 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.546162 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-serving-cert\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.546211 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.546185 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-oauth-serving-cert\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.647517 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.647475 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-service-ca\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.647517 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.647517 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l5pp\" (UniqueName: \"kubernetes.io/projected/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-kube-api-access-2l5pp\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.647769 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.647539 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-oauth-config\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.647769 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.647559 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-serving-cert\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.647769 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.647584 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-oauth-serving-cert\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.647769 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.647630 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-config\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.648272 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.648242 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-service-ca\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.648403 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.648351 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-oauth-serving-cert\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.648403 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.648351 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-config\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.649969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.649946 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-oauth-config\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.650046 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.649956 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-serving-cert\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.654505 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.654483 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l5pp\" (UniqueName: \"kubernetes.io/projected/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-kube-api-access-2l5pp\") pod \"console-7cfd87f5b5-47mdc\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.755302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.755218 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:27.894218 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.894184 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:27.996254 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:27.996225 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:28.031252 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:28.031177 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cfd87f5b5-47mdc"] Sep 06 10:31:28.033854 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:31:28.033829 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328f550a_d2a5_4c9e_aa8f_2fb1201ae30f.slice/crio-607e143c7447feb46483be9dcae0bf36a125a55d5db0fee72511740570821c14 WatchSource:0}: Error finding container 607e143c7447feb46483be9dcae0bf36a125a55d5db0fee72511740570821c14: Status 404 returned error can't find the container with id 607e143c7447feb46483be9dcae0bf36a125a55d5db0fee72511740570821c14 Sep 06 10:31:28.317357 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:28.317275 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cfd87f5b5-47mdc" event={"ID":"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f","Type":"ContainerStarted","Data":"607e143c7447feb46483be9dcae0bf36a125a55d5db0fee72511740570821c14"} Sep 06 10:31:30.320297 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:30.320161 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:31.331800 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:31.331769 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cfd87f5b5-47mdc" event={"ID":"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f","Type":"ContainerStarted","Data":"70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b"} Sep 06 10:31:31.348806 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:31.348752 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cfd87f5b5-47mdc" podStartSLOduration=1.475466389 podStartE2EDuration="4.348737037s" totalImagesPullingTime="2.873270648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:31.348240445 +0000 UTC m=+84.930730087" watchObservedRunningTime="2026-09-06 10:31:31.348737037 +0000 UTC m=+84.931226709" Sep 06 10:31:32.816874 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:32.816838 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:32.917380 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:32.917345 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:32.970841 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:32.970804 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:36.908852 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:36.908815 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54477d4985-ljnv2"] Sep 06 10:31:36.912139 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:36.912101 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:36.918807 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:36.918789 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 06 10:31:36.925657 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:36.925632 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54477d4985-ljnv2"] Sep 06 10:31:37.033246 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.033207 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-oauth-serving-cert\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.033418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.033255 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-trusted-ca-bundle\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.033418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.033284 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-oauth-config\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.033418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.033336 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-serving-cert\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.033418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.033382 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-service-ca\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.033418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.033413 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h9f4\" (UniqueName: \"kubernetes.io/projected/4c9d5b93-1ae2-48c3-9180-743dae4f847e-kube-api-access-7h9f4\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.033577 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.033457 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-config\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.134627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.134571 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-oauth-config\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.134627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.134627 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-serving-cert\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.134838 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.134739 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-service-ca\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.134838 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.134780 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h9f4\" (UniqueName: \"kubernetes.io/projected/4c9d5b93-1ae2-48c3-9180-743dae4f847e-kube-api-access-7h9f4\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.134944 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.134917 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-config\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.135024 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.135007 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-oauth-serving-cert\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.135076 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.135055 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-trusted-ca-bundle\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.135497 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.135471 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-service-ca\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.135625 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.135608 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-config\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.135794 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.135769 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-oauth-serving-cert\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.135853 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.135813 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-trusted-ca-bundle\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.137110 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.137089 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-oauth-config\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.137241 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.137224 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-serving-cert\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.142490 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.142464 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h9f4\" (UniqueName: \"kubernetes.io/projected/4c9d5b93-1ae2-48c3-9180-743dae4f847e-kube-api-access-7h9f4\") pod \"console-54477d4985-ljnv2\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.222176 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.222116 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:37.310442 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.310402 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:37.367374 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.367344 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:37.489438 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.489414 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54477d4985-ljnv2"] Sep 06 10:31:37.492175 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:31:37.492147 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c9d5b93_1ae2_48c3_9180_743dae4f847e.slice/crio-52202db7cd296473363f1ca69e1cc4f2cbef12cc3ec5fb6c4676a2d9055e9df8 WatchSource:0}: Error finding container 52202db7cd296473363f1ca69e1cc4f2cbef12cc3ec5fb6c4676a2d9055e9df8: Status 404 returned error can't find the container with id 52202db7cd296473363f1ca69e1cc4f2cbef12cc3ec5fb6c4676a2d9055e9df8 Sep 06 10:31:37.756011 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.755922 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:37.756011 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.755984 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:37.760518 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:37.760495 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:38.358016 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:38.357977 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54477d4985-ljnv2" event={"ID":"4c9d5b93-1ae2-48c3-9180-743dae4f847e","Type":"ContainerStarted","Data":"607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5"} Sep 06 10:31:38.358422 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:38.358027 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54477d4985-ljnv2" event={"ID":"4c9d5b93-1ae2-48c3-9180-743dae4f847e","Type":"ContainerStarted","Data":"52202db7cd296473363f1ca69e1cc4f2cbef12cc3ec5fb6c4676a2d9055e9df8"} Sep 06 10:31:38.362025 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:38.362001 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:31:38.376657 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:38.376614 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54477d4985-ljnv2" podStartSLOduration=2.376599704 podStartE2EDuration="2.376599704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:31:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:31:38.375664238 +0000 UTC m=+91.958153862" watchObservedRunningTime="2026-09-06 10:31:38.376599704 +0000 UTC m=+91.959089334" Sep 06 10:31:40.430133 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:40.430096 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:40.531468 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:40.531423 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:47.223112 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:47.223047 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:47.223705 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:47.223188 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:47.227830 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:47.227808 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:47.393627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:47.393602 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:31:47.436847 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:47.436820 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cfd87f5b5-47mdc"] Sep 06 10:31:47.959740 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:47.959708 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:48.914396 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:48.914367 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:48.965469 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:48.965427 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:50.286441 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:50.286410 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lrpg9" Sep 06 10:31:55.417852 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:55.417820 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:55.486605 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:55.486570 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:56.337192 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:56.337159 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:31:56.396665 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:31:56.396630 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:00.592769 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:00.592728 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 06 10:32:00.593184 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:00.593022 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="alertmanager" containerID="cri-o://88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704" gracePeriod=120 Sep 06 10:32:00.593184 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:00.593090 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="kube-rbac-proxy-metric" containerID="cri-o://103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300" gracePeriod=120 Sep 06 10:32:00.593184 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:00.593148 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="prom-label-proxy" containerID="cri-o://daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09" gracePeriod=120 Sep 06 10:32:00.593358 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:00.593165 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="kube-rbac-proxy-web" containerID="cri-o://73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971" gracePeriod=120 Sep 06 10:32:00.593358 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:00.593199 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="config-reloader" containerID="cri-o://8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37" gracePeriod=120 Sep 06 10:32:00.593358 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:00.593167 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="kube-rbac-proxy" containerID="cri-o://85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065" gracePeriod=120 Sep 06 10:32:01.444409 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.444372 2574 generic.go:356] "Generic (PLEG): container finished" podID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerID="daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09" exitCode=0 Sep 06 10:32:01.444409 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.444398 2574 generic.go:356] "Generic (PLEG): container finished" podID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerID="85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065" exitCode=0 Sep 06 10:32:01.444409 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.444406 2574 generic.go:356] "Generic (PLEG): container finished" podID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerID="8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37" exitCode=0 Sep 06 10:32:01.444409 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.444413 2574 generic.go:356] "Generic (PLEG): container finished" podID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerID="88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704" exitCode=0 Sep 06 10:32:01.444688 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.444444 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerDied","Data":"daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09"} Sep 06 10:32:01.444688 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.444507 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerDied","Data":"85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065"} Sep 06 10:32:01.444688 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.444533 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerDied","Data":"8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37"} Sep 06 10:32:01.444688 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.444558 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerDied","Data":"88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704"} Sep 06 10:32:01.833984 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.833958 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:01.931606 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931573 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy-web\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.931802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931624 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy-metric\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.931802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931644 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-cluster-tls-config\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.931802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931665 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-main-tls\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.931802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931684 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-config-volume\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.931802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931725 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dxth\" (UniqueName: \"kubernetes.io/projected/db4211e5-0dd2-48bd-a4b8-f891dd452348-kube-api-access-6dxth\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.931802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931746 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-web-config\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.931802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931775 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/db4211e5-0dd2-48bd-a4b8-f891dd452348-config-out\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.931802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931798 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-metrics-client-ca\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.932235 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931860 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-main-db\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.932235 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931893 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.932235 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931920 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/db4211e5-0dd2-48bd-a4b8-f891dd452348-tls-assets\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.932235 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.931946 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-trusted-ca-bundle\") pod \"db4211e5-0dd2-48bd-a4b8-f891dd452348\" (UID: \"db4211e5-0dd2-48bd-a4b8-f891dd452348\") " Sep 06 10:32:01.933332 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.932776 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:32:01.933332 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.932848 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:32:01.933332 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.933310 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:32:01.935243 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.935196 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:32:01.935795 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.935761 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:32:01.935931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.935911 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-config-volume" (OuterVolumeSpecName: "config-volume") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:32:01.936209 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.936179 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:32:01.936309 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.936202 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db4211e5-0dd2-48bd-a4b8-f891dd452348-kube-api-access-6dxth" (OuterVolumeSpecName: "kube-api-access-6dxth") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "kube-api-access-6dxth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:32:01.936309 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.936285 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db4211e5-0dd2-48bd-a4b8-f891dd452348-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:32:01.936418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.936319 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:32:01.936472 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.936458 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db4211e5-0dd2-48bd-a4b8-f891dd452348-config-out" (OuterVolumeSpecName: "config-out") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:32:01.939062 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.939043 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:32:01.945437 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:01.945412 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-web-config" (OuterVolumeSpecName: "web-config") pod "db4211e5-0dd2-48bd-a4b8-f891dd452348" (UID: "db4211e5-0dd2-48bd-a4b8-f891dd452348"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:32:02.032685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032589 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.032685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032633 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/db4211e5-0dd2-48bd-a4b8-f891dd452348-tls-assets\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.032685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032644 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.032685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032656 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.032685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032665 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.032685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032675 2574 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-cluster-tls-config\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.032685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032684 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-secret-alertmanager-main-tls\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.032685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032692 2574 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.032685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032700 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dxth\" (UniqueName: \"kubernetes.io/projected/db4211e5-0dd2-48bd-a4b8-f891dd452348-kube-api-access-6dxth\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.033050 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032730 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/db4211e5-0dd2-48bd-a4b8-f891dd452348-web-config\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.033050 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032738 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/db4211e5-0dd2-48bd-a4b8-f891dd452348-config-out\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.033050 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032745 2574 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db4211e5-0dd2-48bd-a4b8-f891dd452348-metrics-client-ca\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.033050 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.032753 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/db4211e5-0dd2-48bd-a4b8-f891dd452348-alertmanager-main-db\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:02.451627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.451589 2574 generic.go:356] "Generic (PLEG): container finished" podID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerID="103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300" exitCode=0 Sep 06 10:32:02.451627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.451623 2574 generic.go:356] "Generic (PLEG): container finished" podID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerID="73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971" exitCode=0 Sep 06 10:32:02.451833 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.451677 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerDied","Data":"103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300"} Sep 06 10:32:02.451833 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.451698 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.451833 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.451754 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerDied","Data":"73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971"} Sep 06 10:32:02.451833 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.451783 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"db4211e5-0dd2-48bd-a4b8-f891dd452348","Type":"ContainerDied","Data":"4653324f363b2a1f07cec33f7fbb0ba033885fee52eb385efad31836ea216363"} Sep 06 10:32:02.451833 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.451814 2574 scope.go:117] "RemoveContainer" containerID="daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09" Sep 06 10:32:02.460287 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.460265 2574 scope.go:117] "RemoveContainer" containerID="103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300" Sep 06 10:32:02.467325 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.467308 2574 scope.go:117] "RemoveContainer" containerID="85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065" Sep 06 10:32:02.475075 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.475045 2574 scope.go:117] "RemoveContainer" containerID="73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971" Sep 06 10:32:02.475384 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.475365 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 06 10:32:02.481057 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.481031 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 06 10:32:02.484356 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.484341 2574 scope.go:117] "RemoveContainer" containerID="8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37" Sep 06 10:32:02.491005 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.490984 2574 scope.go:117] "RemoveContainer" containerID="88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704" Sep 06 10:32:02.497754 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.497731 2574 scope.go:117] "RemoveContainer" containerID="0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c" Sep 06 10:32:02.504413 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.504396 2574 scope.go:117] "RemoveContainer" containerID="daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09" Sep 06 10:32:02.504661 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:32:02.504644 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09\": container with ID starting with daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09 not found: ID does not exist" containerID="daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09" Sep 06 10:32:02.504708 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.504668 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09"} err="failed to get container status \"daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09\": rpc error: code = NotFound desc = could not find container \"daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09\": container with ID starting with daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09 not found: ID does not exist" Sep 06 10:32:02.504708 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.504707 2574 scope.go:117] "RemoveContainer" containerID="103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300" Sep 06 10:32:02.504945 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:32:02.504927 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300\": container with ID starting with 103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300 not found: ID does not exist" containerID="103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300" Sep 06 10:32:02.505006 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.504955 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300"} err="failed to get container status \"103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300\": rpc error: code = NotFound desc = could not find container \"103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300\": container with ID starting with 103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300 not found: ID does not exist" Sep 06 10:32:02.505006 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.504977 2574 scope.go:117] "RemoveContainer" containerID="85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065" Sep 06 10:32:02.505211 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:32:02.505194 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065\": container with ID starting with 85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065 not found: ID does not exist" containerID="85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065" Sep 06 10:32:02.505257 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.505216 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065"} err="failed to get container status \"85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065\": rpc error: code = NotFound desc = could not find container \"85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065\": container with ID starting with 85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065 not found: ID does not exist" Sep 06 10:32:02.505257 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.505230 2574 scope.go:117] "RemoveContainer" containerID="73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971" Sep 06 10:32:02.505459 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:32:02.505444 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971\": container with ID starting with 73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971 not found: ID does not exist" containerID="73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971" Sep 06 10:32:02.505506 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.505462 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971"} err="failed to get container status \"73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971\": rpc error: code = NotFound desc = could not find container \"73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971\": container with ID starting with 73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971 not found: ID does not exist" Sep 06 10:32:02.505506 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.505475 2574 scope.go:117] "RemoveContainer" containerID="8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37" Sep 06 10:32:02.505672 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:32:02.505657 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37\": container with ID starting with 8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37 not found: ID does not exist" containerID="8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37" Sep 06 10:32:02.505709 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.505674 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37"} err="failed to get container status \"8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37\": rpc error: code = NotFound desc = could not find container \"8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37\": container with ID starting with 8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37 not found: ID does not exist" Sep 06 10:32:02.505709 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.505685 2574 scope.go:117] "RemoveContainer" containerID="88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704" Sep 06 10:32:02.505894 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:32:02.505877 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704\": container with ID starting with 88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704 not found: ID does not exist" containerID="88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704" Sep 06 10:32:02.505955 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.505901 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704"} err="failed to get container status \"88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704\": rpc error: code = NotFound desc = could not find container \"88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704\": container with ID starting with 88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704 not found: ID does not exist" Sep 06 10:32:02.505955 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.505921 2574 scope.go:117] "RemoveContainer" containerID="0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c" Sep 06 10:32:02.506180 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:32:02.506156 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c\": container with ID starting with 0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c not found: ID does not exist" containerID="0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c" Sep 06 10:32:02.506274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.506181 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c"} err="failed to get container status \"0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c\": rpc error: code = NotFound desc = could not find container \"0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c\": container with ID starting with 0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c not found: ID does not exist" Sep 06 10:32:02.506274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.506195 2574 scope.go:117] "RemoveContainer" containerID="daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09" Sep 06 10:32:02.506464 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.506441 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09"} err="failed to get container status \"daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09\": rpc error: code = NotFound desc = could not find container \"daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09\": container with ID starting with daf420a0e60afecef28cae1b4bc40a7cfe9313a33501b967a3749076a10abe09 not found: ID does not exist" Sep 06 10:32:02.506506 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.506465 2574 scope.go:117] "RemoveContainer" containerID="103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300" Sep 06 10:32:02.506710 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.506694 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300"} err="failed to get container status \"103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300\": rpc error: code = NotFound desc = could not find container \"103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300\": container with ID starting with 103e126a7fb9af3d1022f50ae92e741fbf7b439458cc0382dd23a035803df300 not found: ID does not exist" Sep 06 10:32:02.506752 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.506713 2574 scope.go:117] "RemoveContainer" containerID="85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065" Sep 06 10:32:02.506958 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.506942 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065"} err="failed to get container status \"85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065\": rpc error: code = NotFound desc = could not find container \"85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065\": container with ID starting with 85650a8a09a8aed7d841e6870fcf91045d61fac586cbc34f7e286ce538fbd065 not found: ID does not exist" Sep 06 10:32:02.507002 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.506958 2574 scope.go:117] "RemoveContainer" containerID="73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971" Sep 06 10:32:02.507183 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.507166 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971"} err="failed to get container status \"73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971\": rpc error: code = NotFound desc = could not find container \"73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971\": container with ID starting with 73c2c5fac073c156d030fd2dae2e0af946bb6858893b49ea0320f2f61863f971 not found: ID does not exist" Sep 06 10:32:02.507231 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.507183 2574 scope.go:117] "RemoveContainer" containerID="8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37" Sep 06 10:32:02.507410 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.507392 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37"} err="failed to get container status \"8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37\": rpc error: code = NotFound desc = could not find container \"8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37\": container with ID starting with 8aef4f87fdbafec788bda12350fa88a5e4a747fe010221af5eb791e03bc14f37 not found: ID does not exist" Sep 06 10:32:02.507464 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.507410 2574 scope.go:117] "RemoveContainer" containerID="88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704" Sep 06 10:32:02.507696 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.507667 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704"} err="failed to get container status \"88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704\": rpc error: code = NotFound desc = could not find container \"88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704\": container with ID starting with 88e4d91145d909af5f85413f0c1a3d048b370a530c97af154fd63b0290568704 not found: ID does not exist" Sep 06 10:32:02.507696 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.507693 2574 scope.go:117] "RemoveContainer" containerID="0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c" Sep 06 10:32:02.508295 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.508273 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c"} err="failed to get container status \"0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c\": rpc error: code = NotFound desc = could not find container \"0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c\": container with ID starting with 0fdbc3a18f92b76be693886c1dd70b5023b1602a7aaf3d3b8dae21f330897a0c not found: ID does not exist" Sep 06 10:32:02.509368 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509351 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 06 10:32:02.509607 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509597 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="init-config-reloader" Sep 06 10:32:02.509647 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509609 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="init-config-reloader" Sep 06 10:32:02.509647 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509618 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="config-reloader" Sep 06 10:32:02.509647 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509624 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="config-reloader" Sep 06 10:32:02.509647 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509631 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="kube-rbac-proxy" Sep 06 10:32:02.509647 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509636 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="kube-rbac-proxy" Sep 06 10:32:02.509647 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509641 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="prom-label-proxy" Sep 06 10:32:02.509647 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509646 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="prom-label-proxy" Sep 06 10:32:02.509835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509653 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="kube-rbac-proxy-web" Sep 06 10:32:02.509835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509658 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="kube-rbac-proxy-web" Sep 06 10:32:02.509835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509671 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="alertmanager" Sep 06 10:32:02.509835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509676 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="alertmanager" Sep 06 10:32:02.509835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509686 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="kube-rbac-proxy-metric" Sep 06 10:32:02.509835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509691 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="kube-rbac-proxy-metric" Sep 06 10:32:02.509835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509732 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="config-reloader" Sep 06 10:32:02.509835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509738 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="kube-rbac-proxy-web" Sep 06 10:32:02.509835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509745 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="alertmanager" Sep 06 10:32:02.509835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509752 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="kube-rbac-proxy-metric" Sep 06 10:32:02.509835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509759 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="kube-rbac-proxy" Sep 06 10:32:02.509835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.509765 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" containerName="prom-label-proxy" Sep 06 10:32:02.513094 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.513080 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.515687 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.515667 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 06 10:32:02.515784 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.515669 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 06 10:32:02.515784 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.515671 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 06 10:32:02.515913 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.515900 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x6xvm\"" Sep 06 10:32:02.515992 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.515914 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 06 10:32:02.515992 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.515923 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 06 10:32:02.515992 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.515928 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 06 10:32:02.515992 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.515947 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 06 10:32:02.515992 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.515965 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 06 10:32:02.520962 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.520943 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 06 10:32:02.527985 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.527962 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 06 10:32:02.535838 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.535815 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.535941 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.535842 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.535941 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.535868 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.535941 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.535885 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.535941 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.535901 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.536083 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.535944 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-546bf\" (UniqueName: \"kubernetes.io/projected/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-kube-api-access-546bf\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.536083 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.536008 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.536083 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.536061 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.536198 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.536106 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.536198 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.536159 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-web-config\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.536198 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.536186 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.536299 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.536249 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-config-out\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.536299 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.536276 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.636755 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.636716 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.636755 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.636757 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-web-config\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.636969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.636776 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.636969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.636833 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-config-out\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.636969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.636893 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.636969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.636925 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.636969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.636948 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.637254 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.636987 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.637254 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.637011 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.637254 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.637032 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.637254 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.637054 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-546bf\" (UniqueName: \"kubernetes.io/projected/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-kube-api-access-546bf\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.637254 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.637082 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.637254 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.637147 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.637533 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.637485 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.638449 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.638309 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.638789 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.638763 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.639834 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.639808 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.640274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.640248 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-web-config\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.640364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.640252 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.640364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.640255 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.640364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.640345 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.640563 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.640546 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-config-volume\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.640898 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.640883 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.641035 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.641015 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.641439 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.641421 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-config-out\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.649967 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.649949 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-546bf\" (UniqueName: \"kubernetes.io/projected/42e1d522-2daf-4a64-8f7a-f2f95fa428bb-kube-api-access-546bf\") pod \"alertmanager-main-0\" (UID: \"42e1d522-2daf-4a64-8f7a-f2f95fa428bb\") " pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.822364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.822260 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 06 10:32:02.925243 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:02.925212 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db4211e5-0dd2-48bd-a4b8-f891dd452348" path="/var/lib/kubelet/pods/db4211e5-0dd2-48bd-a4b8-f891dd452348/volumes" Sep 06 10:32:03.103407 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:03.103384 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 06 10:32:03.458296 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:03.458258 2574 generic.go:356] "Generic (PLEG): container finished" podID="42e1d522-2daf-4a64-8f7a-f2f95fa428bb" containerID="924236d476de7c0625529a57d28dc44cdca3151b621e7b3cb4aab47133f80377" exitCode=0 Sep 06 10:32:03.458461 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:03.458308 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42e1d522-2daf-4a64-8f7a-f2f95fa428bb","Type":"ContainerDied","Data":"924236d476de7c0625529a57d28dc44cdca3151b621e7b3cb4aab47133f80377"} Sep 06 10:32:03.458461 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:03.458400 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42e1d522-2daf-4a64-8f7a-f2f95fa428bb","Type":"ContainerStarted","Data":"1614d533b44d0f27fc89388a19df3b70bfa4a0479add7c8501da2d8087bea87c"} Sep 06 10:32:04.467340 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:04.467305 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42e1d522-2daf-4a64-8f7a-f2f95fa428bb","Type":"ContainerStarted","Data":"9b3a0ccd8ed546f8806851700f3dd68fb094421ee219cbd2c5c22d65373bf52b"} Sep 06 10:32:04.467822 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:04.467389 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42e1d522-2daf-4a64-8f7a-f2f95fa428bb","Type":"ContainerStarted","Data":"95bf8bd4720dcbefa3beaa487a7ce07d698eab85e1bf1c5d054a5ca2ff71da8d"} Sep 06 10:32:04.467822 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:04.467436 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42e1d522-2daf-4a64-8f7a-f2f95fa428bb","Type":"ContainerStarted","Data":"4d11886603489b84ff78d71f9fe27e3cf76621574f8d56e1ae70f703df0bbdaa"} Sep 06 10:32:04.467822 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:04.467487 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42e1d522-2daf-4a64-8f7a-f2f95fa428bb","Type":"ContainerStarted","Data":"8cb3b9d8e7433ab359fdf0e9c9fd38b4951d0f2d060ee2dd37df854ed0c558d9"} Sep 06 10:32:04.467822 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:04.467537 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42e1d522-2daf-4a64-8f7a-f2f95fa428bb","Type":"ContainerStarted","Data":"d73a7e6af4ea9ac8b436cf046f019fe360334d27a7cbb7e06f4b26be956b699b"} Sep 06 10:32:04.467822 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:04.467576 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"42e1d522-2daf-4a64-8f7a-f2f95fa428bb","Type":"ContainerStarted","Data":"1183ba8435b6d9cfd8b6bc2122a9d09f495a79cbdccd7e54305483cd33c5f079"} Sep 06 10:32:04.495002 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:04.494951 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.47825899 podStartE2EDuration="2.49493372s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.67473ms" isStatefulPod=false podCreationTimestamp="2026-09-06 10:32:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:32:04.493437926 +0000 UTC m=+118.075927557" watchObservedRunningTime="2026-09-06 10:32:04.49493372 +0000 UTC m=+118.077423351" Sep 06 10:32:10.421984 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:10.421949 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:10.533371 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:10.533333 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:12.456821 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.456777 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cfd87f5b5-47mdc" podUID="328f550a-d2a5-4c9e-aa8f-2fb1201ae30f" containerName="console" containerID="cri-o://70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b" gracePeriod=15 Sep 06 10:32:12.683854 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.683832 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:32:12.722023 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.721942 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-config\") pod \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " Sep 06 10:32:12.722023 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.721982 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-oauth-config\") pod \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " Sep 06 10:32:12.722023 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.722012 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-serving-cert\") pod \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " Sep 06 10:32:12.722292 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.722062 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-oauth-serving-cert\") pod \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " Sep 06 10:32:12.722292 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.722117 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-service-ca\") pod \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " Sep 06 10:32:12.722292 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.722175 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l5pp\" (UniqueName: \"kubernetes.io/projected/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-kube-api-access-2l5pp\") pod \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\" (UID: \"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f\") " Sep 06 10:32:12.722475 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.722450 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-config" (OuterVolumeSpecName: "console-config") pod "328f550a-d2a5-4c9e-aa8f-2fb1201ae30f" (UID: "328f550a-d2a5-4c9e-aa8f-2fb1201ae30f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:32:12.722585 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.722509 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-service-ca" (OuterVolumeSpecName: "service-ca") pod "328f550a-d2a5-4c9e-aa8f-2fb1201ae30f" (UID: "328f550a-d2a5-4c9e-aa8f-2fb1201ae30f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:32:12.722585 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.722538 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "328f550a-d2a5-4c9e-aa8f-2fb1201ae30f" (UID: "328f550a-d2a5-4c9e-aa8f-2fb1201ae30f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:32:12.724364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.724342 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "328f550a-d2a5-4c9e-aa8f-2fb1201ae30f" (UID: "328f550a-d2a5-4c9e-aa8f-2fb1201ae30f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:32:12.724496 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.724472 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-kube-api-access-2l5pp" (OuterVolumeSpecName: "kube-api-access-2l5pp") pod "328f550a-d2a5-4c9e-aa8f-2fb1201ae30f" (UID: "328f550a-d2a5-4c9e-aa8f-2fb1201ae30f"). InnerVolumeSpecName "kube-api-access-2l5pp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:32:12.724536 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.724499 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "328f550a-d2a5-4c9e-aa8f-2fb1201ae30f" (UID: "328f550a-d2a5-4c9e-aa8f-2fb1201ae30f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:32:12.823382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.823317 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-oauth-serving-cert\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:12.823382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.823370 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-service-ca\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:12.823382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.823379 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l5pp\" (UniqueName: \"kubernetes.io/projected/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-kube-api-access-2l5pp\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:12.823382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.823389 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-config\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:12.823382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.823399 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-oauth-config\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:12.823685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:12.823407 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f-console-serving-cert\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:13.282693 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:13.282658 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:13.346319 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:13.346271 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:13.506802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:13.506770 2574 generic.go:356] "Generic (PLEG): container finished" podID="328f550a-d2a5-4c9e-aa8f-2fb1201ae30f" containerID="70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b" exitCode=0 Sep 06 10:32:13.507220 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:13.506812 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cfd87f5b5-47mdc" Sep 06 10:32:13.507220 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:13.506841 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cfd87f5b5-47mdc" event={"ID":"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f","Type":"ContainerDied","Data":"70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b"} Sep 06 10:32:13.507220 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:13.506889 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cfd87f5b5-47mdc" event={"ID":"328f550a-d2a5-4c9e-aa8f-2fb1201ae30f","Type":"ContainerDied","Data":"607e143c7447feb46483be9dcae0bf36a125a55d5db0fee72511740570821c14"} Sep 06 10:32:13.507220 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:13.506922 2574 scope.go:117] "RemoveContainer" containerID="70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b" Sep 06 10:32:13.514916 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:13.514897 2574 scope.go:117] "RemoveContainer" containerID="70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b" Sep 06 10:32:13.515199 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:32:13.515171 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b\": container with ID starting with 70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b not found: ID does not exist" containerID="70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b" Sep 06 10:32:13.515304 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:13.515207 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b"} err="failed to get container status \"70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b\": rpc error: code = NotFound desc = could not find container \"70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b\": container with ID starting with 70f042115a8d2823320dcf1f81ae1dcbec1730b88f53762c7a7c92463d978b6b not found: ID does not exist" Sep 06 10:32:13.523116 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:13.523088 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cfd87f5b5-47mdc"] Sep 06 10:32:13.527093 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:13.527070 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cfd87f5b5-47mdc"] Sep 06 10:32:14.924794 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:14.924761 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="328f550a-d2a5-4c9e-aa8f-2fb1201ae30f" path="/var/lib/kubelet/pods/328f550a-d2a5-4c9e-aa8f-2fb1201ae30f/volumes" Sep 06 10:32:16.014349 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.014305 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7656c95999-6pvms"] Sep 06 10:32:16.014846 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.014590 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="328f550a-d2a5-4c9e-aa8f-2fb1201ae30f" containerName="console" Sep 06 10:32:16.014846 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.014602 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="328f550a-d2a5-4c9e-aa8f-2fb1201ae30f" containerName="console" Sep 06 10:32:16.014846 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.014655 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="328f550a-d2a5-4c9e-aa8f-2fb1201ae30f" containerName="console" Sep 06 10:32:16.018976 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.018956 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.026445 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.026420 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7656c95999-6pvms"] Sep 06 10:32:16.051058 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.051019 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-serving-cert\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.051058 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.051061 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-oauth-config\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.051300 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.051093 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-trusted-ca-bundle\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.051300 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.051145 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-service-ca\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.051300 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.051175 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-config\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.051300 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.051195 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-oauth-serving-cert\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.051300 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.051227 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf69m\" (UniqueName: \"kubernetes.io/projected/d0c3c91e-aeb1-4843-a8ab-0882781d767f-kube-api-access-xf69m\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.098979 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.098952 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:16.152215 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.152177 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-serving-cert\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.152215 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.152215 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-oauth-config\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.152437 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.152249 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-trusted-ca-bundle\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.152437 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.152282 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-service-ca\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.152437 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.152309 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-config\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.152437 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.152332 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-oauth-serving-cert\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.152437 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.152356 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf69m\" (UniqueName: \"kubernetes.io/projected/d0c3c91e-aeb1-4843-a8ab-0882781d767f-kube-api-access-xf69m\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.153153 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.153110 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-service-ca\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.153153 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.153113 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-oauth-serving-cert\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.153348 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.153113 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-config\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.153348 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.153318 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-trusted-ca-bundle\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.154871 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.154844 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-serving-cert\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.154980 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.154880 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-oauth-config\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.155100 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.155082 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:16.159917 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.159892 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf69m\" (UniqueName: \"kubernetes.io/projected/d0c3c91e-aeb1-4843-a8ab-0882781d767f-kube-api-access-xf69m\") pod \"console-7656c95999-6pvms\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.327765 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.327666 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:16.598255 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:16.598229 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7656c95999-6pvms"] Sep 06 10:32:16.600670 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:32:16.600637 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c3c91e_aeb1_4843_a8ab_0882781d767f.slice/crio-fbaccbd45b18007a8d3fc9202c450820526a1166ebb63c13428df059c6328ae6 WatchSource:0}: Error finding container fbaccbd45b18007a8d3fc9202c450820526a1166ebb63c13428df059c6328ae6: Status 404 returned error can't find the container with id fbaccbd45b18007a8d3fc9202c450820526a1166ebb63c13428df059c6328ae6 Sep 06 10:32:17.523286 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:17.523250 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7656c95999-6pvms" event={"ID":"d0c3c91e-aeb1-4843-a8ab-0882781d767f","Type":"ContainerStarted","Data":"af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae"} Sep 06 10:32:17.523715 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:17.523300 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7656c95999-6pvms" event={"ID":"d0c3c91e-aeb1-4843-a8ab-0882781d767f","Type":"ContainerStarted","Data":"fbaccbd45b18007a8d3fc9202c450820526a1166ebb63c13428df059c6328ae6"} Sep 06 10:32:17.540494 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:17.540443 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7656c95999-6pvms" podStartSLOduration=2.540425357 podStartE2EDuration="2.540425357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:32:17.539738447 +0000 UTC m=+131.122228104" watchObservedRunningTime="2026-09-06 10:32:17.540425357 +0000 UTC m=+131.122914990" Sep 06 10:32:21.284741 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:21.284707 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:21.386142 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:21.386101 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:26.328099 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:26.328061 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:26.328502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:26.328137 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:26.332862 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:26.332833 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:26.558842 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:26.558807 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:32:26.602662 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:26.602589 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54477d4985-ljnv2"] Sep 06 10:32:26.692837 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:26.692809 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:26.797444 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:26.797409 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:26.854114 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:26.854037 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:29.554306 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:29.554259 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:32:51.623985 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:51.623925 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54477d4985-ljnv2" podUID="4c9d5b93-1ae2-48c3-9180-743dae4f847e" containerName="console" containerID="cri-o://607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5" gracePeriod=15 Sep 06 10:32:51.862539 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:51.862514 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:32:52.047953 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.047912 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-trusted-ca-bundle\") pod \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " Sep 06 10:32:52.047953 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.047956 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-config\") pod \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " Sep 06 10:32:52.048237 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.047977 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-oauth-serving-cert\") pod \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " Sep 06 10:32:52.048237 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.048007 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-service-ca\") pod \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " Sep 06 10:32:52.048237 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.048037 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-serving-cert\") pod \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " Sep 06 10:32:52.048237 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.048064 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-oauth-config\") pod \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " Sep 06 10:32:52.048237 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.048149 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h9f4\" (UniqueName: \"kubernetes.io/projected/4c9d5b93-1ae2-48c3-9180-743dae4f847e-kube-api-access-7h9f4\") pod \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\" (UID: \"4c9d5b93-1ae2-48c3-9180-743dae4f847e\") " Sep 06 10:32:52.048477 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.048423 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-config" (OuterVolumeSpecName: "console-config") pod "4c9d5b93-1ae2-48c3-9180-743dae4f847e" (UID: "4c9d5b93-1ae2-48c3-9180-743dae4f847e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:32:52.048477 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.048462 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4c9d5b93-1ae2-48c3-9180-743dae4f847e" (UID: "4c9d5b93-1ae2-48c3-9180-743dae4f847e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:32:52.048575 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.048479 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4c9d5b93-1ae2-48c3-9180-743dae4f847e" (UID: "4c9d5b93-1ae2-48c3-9180-743dae4f847e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:32:52.048575 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.048458 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-service-ca" (OuterVolumeSpecName: "service-ca") pod "4c9d5b93-1ae2-48c3-9180-743dae4f847e" (UID: "4c9d5b93-1ae2-48c3-9180-743dae4f847e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:32:52.050343 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.050319 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4c9d5b93-1ae2-48c3-9180-743dae4f847e" (UID: "4c9d5b93-1ae2-48c3-9180-743dae4f847e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:32:52.050343 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.050335 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4c9d5b93-1ae2-48c3-9180-743dae4f847e" (UID: "4c9d5b93-1ae2-48c3-9180-743dae4f847e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:32:52.050487 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.050441 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c9d5b93-1ae2-48c3-9180-743dae4f847e-kube-api-access-7h9f4" (OuterVolumeSpecName: "kube-api-access-7h9f4") pod "4c9d5b93-1ae2-48c3-9180-743dae4f847e" (UID: "4c9d5b93-1ae2-48c3-9180-743dae4f847e"). InnerVolumeSpecName "kube-api-access-7h9f4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:32:52.149109 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.149072 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-oauth-config\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:52.149109 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.149102 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h9f4\" (UniqueName: \"kubernetes.io/projected/4c9d5b93-1ae2-48c3-9180-743dae4f847e-kube-api-access-7h9f4\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:52.149109 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.149116 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-trusted-ca-bundle\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:52.149354 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.149145 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-config\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:52.149354 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.149157 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-oauth-serving-cert\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:52.149354 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.149169 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c9d5b93-1ae2-48c3-9180-743dae4f847e-service-ca\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:52.149354 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.149180 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c9d5b93-1ae2-48c3-9180-743dae4f847e-console-serving-cert\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:32:52.648104 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.648071 2574 generic.go:356] "Generic (PLEG): container finished" podID="4c9d5b93-1ae2-48c3-9180-743dae4f847e" containerID="607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5" exitCode=0 Sep 06 10:32:52.648536 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.648136 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54477d4985-ljnv2" Sep 06 10:32:52.648536 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.648140 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54477d4985-ljnv2" event={"ID":"4c9d5b93-1ae2-48c3-9180-743dae4f847e","Type":"ContainerDied","Data":"607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5"} Sep 06 10:32:52.648536 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.648249 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54477d4985-ljnv2" event={"ID":"4c9d5b93-1ae2-48c3-9180-743dae4f847e","Type":"ContainerDied","Data":"52202db7cd296473363f1ca69e1cc4f2cbef12cc3ec5fb6c4676a2d9055e9df8"} Sep 06 10:32:52.648536 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.648269 2574 scope.go:117] "RemoveContainer" containerID="607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5" Sep 06 10:32:52.658409 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.658394 2574 scope.go:117] "RemoveContainer" containerID="607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5" Sep 06 10:32:52.658665 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:32:52.658647 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5\": container with ID starting with 607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5 not found: ID does not exist" containerID="607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5" Sep 06 10:32:52.658719 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.658671 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5"} err="failed to get container status \"607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5\": rpc error: code = NotFound desc = could not find container \"607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5\": container with ID starting with 607106abf429fb0ca5d888831852f8ef6274d7f8e0bd54187c73bce626ffc3e5 not found: ID does not exist" Sep 06 10:32:52.668521 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.668499 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54477d4985-ljnv2"] Sep 06 10:32:52.671903 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.671877 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54477d4985-ljnv2"] Sep 06 10:32:52.925331 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:32:52.925302 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c9d5b93-1ae2-48c3-9180-743dae4f847e" path="/var/lib/kubelet/pods/4c9d5b93-1ae2-48c3-9180-743dae4f847e/volumes" Sep 06 10:33:10.185942 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:10.185895 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:10.285896 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:10.285866 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:10.972599 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:10.972567 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5"] Sep 06 10:33:10.972844 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:10.972831 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c9d5b93-1ae2-48c3-9180-743dae4f847e" containerName="console" Sep 06 10:33:10.972888 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:10.972846 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c9d5b93-1ae2-48c3-9180-743dae4f847e" containerName="console" Sep 06 10:33:10.972943 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:10.972933 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c9d5b93-1ae2-48c3-9180-743dae4f847e" containerName="console" Sep 06 10:33:10.977292 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:10.977275 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:10.980055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:10.980032 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 06 10:33:10.980176 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:10.980075 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 06 10:33:10.980246 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:10.980226 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jqnm5\"" Sep 06 10:33:10.984432 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:10.984406 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5"] Sep 06 10:33:11.115489 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:11.115449 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71ed71c5-21cf-4991-8bab-1a674e48fa4f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5\" (UID: \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:11.115658 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:11.115506 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h45pp\" (UniqueName: \"kubernetes.io/projected/71ed71c5-21cf-4991-8bab-1a674e48fa4f-kube-api-access-h45pp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5\" (UID: \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:11.115658 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:11.115524 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71ed71c5-21cf-4991-8bab-1a674e48fa4f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5\" (UID: \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:11.216476 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:11.216446 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h45pp\" (UniqueName: \"kubernetes.io/projected/71ed71c5-21cf-4991-8bab-1a674e48fa4f-kube-api-access-h45pp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5\" (UID: \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:11.216476 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:11.216479 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71ed71c5-21cf-4991-8bab-1a674e48fa4f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5\" (UID: \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:11.216945 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:11.216532 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71ed71c5-21cf-4991-8bab-1a674e48fa4f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5\" (UID: \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:11.216945 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:11.216826 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71ed71c5-21cf-4991-8bab-1a674e48fa4f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5\" (UID: \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:11.216945 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:11.216915 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71ed71c5-21cf-4991-8bab-1a674e48fa4f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5\" (UID: \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:11.225002 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:11.224941 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h45pp\" (UniqueName: \"kubernetes.io/projected/71ed71c5-21cf-4991-8bab-1a674e48fa4f-kube-api-access-h45pp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5\" (UID: \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:11.287021 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:11.286994 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:11.556283 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:11.556259 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5"] Sep 06 10:33:11.558426 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:33:11.558396 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ed71c5_21cf_4991_8bab_1a674e48fa4f.slice/crio-2361f6ce2935bb912af32298c510d302476a1c5b0f6cfedcd2f0f46a1208605b WatchSource:0}: Error finding container 2361f6ce2935bb912af32298c510d302476a1c5b0f6cfedcd2f0f46a1208605b: Status 404 returned error can't find the container with id 2361f6ce2935bb912af32298c510d302476a1c5b0f6cfedcd2f0f46a1208605b Sep 06 10:33:11.720550 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:11.720516 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" event={"ID":"71ed71c5-21cf-4991-8bab-1a674e48fa4f","Type":"ContainerStarted","Data":"2361f6ce2935bb912af32298c510d302476a1c5b0f6cfedcd2f0f46a1208605b"} Sep 06 10:33:18.747821 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:18.747778 2574 generic.go:356] "Generic (PLEG): container finished" podID="71ed71c5-21cf-4991-8bab-1a674e48fa4f" containerID="863249d8eba3e1746eed4e319058fcbb464449c5776d8eb33ac53f146925ebbe" exitCode=0 Sep 06 10:33:18.748220 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:18.747847 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" event={"ID":"71ed71c5-21cf-4991-8bab-1a674e48fa4f","Type":"ContainerDied","Data":"863249d8eba3e1746eed4e319058fcbb464449c5776d8eb33ac53f146925ebbe"} Sep 06 10:33:21.760807 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:21.760767 2574 generic.go:356] "Generic (PLEG): container finished" podID="71ed71c5-21cf-4991-8bab-1a674e48fa4f" containerID="3feb8ce870ccaf710c41e31dda33ee0460637418703796b7cc903b5b459eac18" exitCode=0 Sep 06 10:33:21.761226 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:21.760821 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" event={"ID":"71ed71c5-21cf-4991-8bab-1a674e48fa4f","Type":"ContainerDied","Data":"3feb8ce870ccaf710c41e31dda33ee0460637418703796b7cc903b5b459eac18"} Sep 06 10:33:23.971018 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:23.970985 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:24.029615 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:24.029580 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:28.791137 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:28.791095 2574 generic.go:356] "Generic (PLEG): container finished" podID="71ed71c5-21cf-4991-8bab-1a674e48fa4f" containerID="d1515c84bab45d2f3c6311d4034b42590d2545d3d9b48eebf10ba94f22b654bc" exitCode=0 Sep 06 10:33:28.791483 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:28.791161 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" event={"ID":"71ed71c5-21cf-4991-8bab-1a674e48fa4f","Type":"ContainerDied","Data":"d1515c84bab45d2f3c6311d4034b42590d2545d3d9b48eebf10ba94f22b654bc"} Sep 06 10:33:29.917776 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:29.917752 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:30.079717 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:30.079634 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71ed71c5-21cf-4991-8bab-1a674e48fa4f-bundle\") pod \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\" (UID: \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\") " Sep 06 10:33:30.079717 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:30.079674 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71ed71c5-21cf-4991-8bab-1a674e48fa4f-util\") pod \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\" (UID: \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\") " Sep 06 10:33:30.079917 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:30.079749 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h45pp\" (UniqueName: \"kubernetes.io/projected/71ed71c5-21cf-4991-8bab-1a674e48fa4f-kube-api-access-h45pp\") pod \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\" (UID: \"71ed71c5-21cf-4991-8bab-1a674e48fa4f\") " Sep 06 10:33:30.080280 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:30.080255 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71ed71c5-21cf-4991-8bab-1a674e48fa4f-bundle" (OuterVolumeSpecName: "bundle") pod "71ed71c5-21cf-4991-8bab-1a674e48fa4f" (UID: "71ed71c5-21cf-4991-8bab-1a674e48fa4f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:33:30.081977 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:30.081949 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71ed71c5-21cf-4991-8bab-1a674e48fa4f-kube-api-access-h45pp" (OuterVolumeSpecName: "kube-api-access-h45pp") pod "71ed71c5-21cf-4991-8bab-1a674e48fa4f" (UID: "71ed71c5-21cf-4991-8bab-1a674e48fa4f"). InnerVolumeSpecName "kube-api-access-h45pp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:33:30.083999 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:30.083975 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71ed71c5-21cf-4991-8bab-1a674e48fa4f-util" (OuterVolumeSpecName: "util") pod "71ed71c5-21cf-4991-8bab-1a674e48fa4f" (UID: "71ed71c5-21cf-4991-8bab-1a674e48fa4f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:33:30.180731 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:30.180707 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h45pp\" (UniqueName: \"kubernetes.io/projected/71ed71c5-21cf-4991-8bab-1a674e48fa4f-kube-api-access-h45pp\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:33:30.180731 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:30.180728 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71ed71c5-21cf-4991-8bab-1a674e48fa4f-bundle\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:33:30.180731 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:30.180738 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71ed71c5-21cf-4991-8bab-1a674e48fa4f-util\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:33:30.801795 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:30.801756 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" event={"ID":"71ed71c5-21cf-4991-8bab-1a674e48fa4f","Type":"ContainerDied","Data":"2361f6ce2935bb912af32298c510d302476a1c5b0f6cfedcd2f0f46a1208605b"} Sep 06 10:33:30.801953 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:30.801809 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2361f6ce2935bb912af32298c510d302476a1c5b0f6cfedcd2f0f46a1208605b" Sep 06 10:33:30.801953 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:30.801766 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qpd5" Sep 06 10:33:31.764983 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:31.764952 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:33.132942 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.132906 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:33.714327 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.714285 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29"] Sep 06 10:33:33.714606 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.714587 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71ed71c5-21cf-4991-8bab-1a674e48fa4f" containerName="util" Sep 06 10:33:33.714606 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.714603 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="71ed71c5-21cf-4991-8bab-1a674e48fa4f" containerName="util" Sep 06 10:33:33.714686 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.714614 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71ed71c5-21cf-4991-8bab-1a674e48fa4f" containerName="pull" Sep 06 10:33:33.714686 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.714620 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="71ed71c5-21cf-4991-8bab-1a674e48fa4f" containerName="pull" Sep 06 10:33:33.714686 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.714635 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71ed71c5-21cf-4991-8bab-1a674e48fa4f" containerName="extract" Sep 06 10:33:33.714686 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.714641 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="71ed71c5-21cf-4991-8bab-1a674e48fa4f" containerName="extract" Sep 06 10:33:33.714686 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.714686 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="71ed71c5-21cf-4991-8bab-1a674e48fa4f" containerName="extract" Sep 06 10:33:33.724376 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.724351 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29" Sep 06 10:33:33.727039 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.727012 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 06 10:33:33.727211 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.727185 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-g62dr\"" Sep 06 10:33:33.727345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.727266 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 06 10:33:33.734348 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.734317 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29"] Sep 06 10:33:33.910966 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.910917 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cebd9cbf-573d-48c0-af50-8ba4a3cd94f3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-w6j29\" (UID: \"cebd9cbf-573d-48c0-af50-8ba4a3cd94f3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29" Sep 06 10:33:33.910966 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:33.910970 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l979k\" (UniqueName: \"kubernetes.io/projected/cebd9cbf-573d-48c0-af50-8ba4a3cd94f3-kube-api-access-l979k\") pod \"cert-manager-operator-controller-manager-55c444bf86-w6j29\" (UID: \"cebd9cbf-573d-48c0-af50-8ba4a3cd94f3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29" Sep 06 10:33:34.012236 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:34.012149 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cebd9cbf-573d-48c0-af50-8ba4a3cd94f3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-w6j29\" (UID: \"cebd9cbf-573d-48c0-af50-8ba4a3cd94f3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29" Sep 06 10:33:34.012236 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:34.012191 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l979k\" (UniqueName: \"kubernetes.io/projected/cebd9cbf-573d-48c0-af50-8ba4a3cd94f3-kube-api-access-l979k\") pod \"cert-manager-operator-controller-manager-55c444bf86-w6j29\" (UID: \"cebd9cbf-573d-48c0-af50-8ba4a3cd94f3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29" Sep 06 10:33:34.012520 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:34.012498 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cebd9cbf-573d-48c0-af50-8ba4a3cd94f3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-w6j29\" (UID: \"cebd9cbf-573d-48c0-af50-8ba4a3cd94f3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29" Sep 06 10:33:34.021026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:34.020994 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l979k\" (UniqueName: \"kubernetes.io/projected/cebd9cbf-573d-48c0-af50-8ba4a3cd94f3-kube-api-access-l979k\") pod \"cert-manager-operator-controller-manager-55c444bf86-w6j29\" (UID: \"cebd9cbf-573d-48c0-af50-8ba4a3cd94f3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29" Sep 06 10:33:34.033789 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:34.033766 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29" Sep 06 10:33:34.337017 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:34.336991 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29"] Sep 06 10:33:34.338846 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:33:34.338823 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcebd9cbf_573d_48c0_af50_8ba4a3cd94f3.slice/crio-89859b3309669e2e0e7d541557d09b0bc0e6c3786cce1f48b7097e814866a5d9 WatchSource:0}: Error finding container 89859b3309669e2e0e7d541557d09b0bc0e6c3786cce1f48b7097e814866a5d9: Status 404 returned error can't find the container with id 89859b3309669e2e0e7d541557d09b0bc0e6c3786cce1f48b7097e814866a5d9 Sep 06 10:33:34.817749 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:34.817710 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29" event={"ID":"cebd9cbf-573d-48c0-af50-8ba4a3cd94f3","Type":"ContainerStarted","Data":"89859b3309669e2e0e7d541557d09b0bc0e6c3786cce1f48b7097e814866a5d9"} Sep 06 10:33:35.385237 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:35.385196 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:35.486709 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:35.486669 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:36.828358 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:36.828268 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29" event={"ID":"cebd9cbf-573d-48c0-af50-8ba4a3cd94f3","Type":"ContainerStarted","Data":"ae40c4f38cb45dd9331bed9f3c7e08e3653b47461c48299b465296286d874d59"} Sep 06 10:33:36.851204 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:36.851151 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w6j29" podStartSLOduration=1.63484468 podStartE2EDuration="3.851113736s" totalImagesPullingTime="2.216269056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:33:36.84871548 +0000 UTC m=+210.431205113" watchObservedRunningTime="2026-09-06 10:33:36.851113736 +0000 UTC m=+210.433603366" Sep 06 10:33:37.172545 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.172517 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:37.271359 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.271326 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:37.451990 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.451912 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:37.572825 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.572791 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:37.786356 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.786278 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xkp5w"] Sep 06 10:33:37.809368 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.809343 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xkp5w"] Sep 06 10:33:37.809514 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.809429 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" Sep 06 10:33:37.811795 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.811768 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 06 10:33:37.811915 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.811903 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 06 10:33:37.812694 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.812669 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-j2vrl\"" Sep 06 10:33:37.840133 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.840091 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fcd0409-d67e-440a-8cf3-d438fe8e8cc9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xkp5w\" (UID: \"4fcd0409-d67e-440a-8cf3-d438fe8e8cc9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" Sep 06 10:33:37.840447 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.840264 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhv2d\" (UniqueName: \"kubernetes.io/projected/4fcd0409-d67e-440a-8cf3-d438fe8e8cc9-kube-api-access-hhv2d\") pod \"cert-manager-webhook-6486ffd997-xkp5w\" (UID: \"4fcd0409-d67e-440a-8cf3-d438fe8e8cc9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" Sep 06 10:33:37.848442 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.848421 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m"] Sep 06 10:33:37.875444 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.875420 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m"] Sep 06 10:33:37.875555 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.875521 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:37.878093 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.878051 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 06 10:33:37.878093 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.878055 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 06 10:33:37.878257 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.878182 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jqnm5\"" Sep 06 10:33:37.941505 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.941471 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhv2d\" (UniqueName: \"kubernetes.io/projected/4fcd0409-d67e-440a-8cf3-d438fe8e8cc9-kube-api-access-hhv2d\") pod \"cert-manager-webhook-6486ffd997-xkp5w\" (UID: \"4fcd0409-d67e-440a-8cf3-d438fe8e8cc9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" Sep 06 10:33:37.941505 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.941515 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d231825d-cbc1-4119-b092-a7faf0797b28-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m\" (UID: \"d231825d-cbc1-4119-b092-a7faf0797b28\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:37.941722 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.941532 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d231825d-cbc1-4119-b092-a7faf0797b28-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m\" (UID: \"d231825d-cbc1-4119-b092-a7faf0797b28\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:37.941722 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.941559 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fcd0409-d67e-440a-8cf3-d438fe8e8cc9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xkp5w\" (UID: \"4fcd0409-d67e-440a-8cf3-d438fe8e8cc9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" Sep 06 10:33:37.941722 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.941707 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbr5n\" (UniqueName: \"kubernetes.io/projected/d231825d-cbc1-4119-b092-a7faf0797b28-kube-api-access-hbr5n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m\" (UID: \"d231825d-cbc1-4119-b092-a7faf0797b28\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:37.949062 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.949031 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fcd0409-d67e-440a-8cf3-d438fe8e8cc9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xkp5w\" (UID: \"4fcd0409-d67e-440a-8cf3-d438fe8e8cc9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" Sep 06 10:33:37.949415 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:37.949396 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhv2d\" (UniqueName: \"kubernetes.io/projected/4fcd0409-d67e-440a-8cf3-d438fe8e8cc9-kube-api-access-hhv2d\") pod \"cert-manager-webhook-6486ffd997-xkp5w\" (UID: \"4fcd0409-d67e-440a-8cf3-d438fe8e8cc9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" Sep 06 10:33:38.042802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.042716 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d231825d-cbc1-4119-b092-a7faf0797b28-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m\" (UID: \"d231825d-cbc1-4119-b092-a7faf0797b28\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:38.042802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.042758 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d231825d-cbc1-4119-b092-a7faf0797b28-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m\" (UID: \"d231825d-cbc1-4119-b092-a7faf0797b28\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:38.042977 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.042824 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbr5n\" (UniqueName: \"kubernetes.io/projected/d231825d-cbc1-4119-b092-a7faf0797b28-kube-api-access-hbr5n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m\" (UID: \"d231825d-cbc1-4119-b092-a7faf0797b28\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:38.043204 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.043184 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d231825d-cbc1-4119-b092-a7faf0797b28-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m\" (UID: \"d231825d-cbc1-4119-b092-a7faf0797b28\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:38.043245 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.043206 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d231825d-cbc1-4119-b092-a7faf0797b28-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m\" (UID: \"d231825d-cbc1-4119-b092-a7faf0797b28\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:38.054899 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.054874 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbr5n\" (UniqueName: \"kubernetes.io/projected/d231825d-cbc1-4119-b092-a7faf0797b28-kube-api-access-hbr5n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m\" (UID: \"d231825d-cbc1-4119-b092-a7faf0797b28\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:38.135777 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.135741 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" Sep 06 10:33:38.184886 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.184851 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:38.428513 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.428486 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xkp5w"] Sep 06 10:33:38.431010 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:33:38.430974 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fcd0409_d67e_440a_8cf3_d438fe8e8cc9.slice/crio-c43ea3de8e08cdb6c3957bd17abb49895024140576507b07eeb2b6bb3a7ad6f6 WatchSource:0}: Error finding container c43ea3de8e08cdb6c3957bd17abb49895024140576507b07eeb2b6bb3a7ad6f6: Status 404 returned error can't find the container with id c43ea3de8e08cdb6c3957bd17abb49895024140576507b07eeb2b6bb3a7ad6f6 Sep 06 10:33:38.473964 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.473938 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m"] Sep 06 10:33:38.476238 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:33:38.476213 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd231825d_cbc1_4119_b092_a7faf0797b28.slice/crio-9cb54c9f5076a5f9b62359e4e678dee6d6dac3802e82cfa9f6a415162b3d2fd1 WatchSource:0}: Error finding container 9cb54c9f5076a5f9b62359e4e678dee6d6dac3802e82cfa9f6a415162b3d2fd1: Status 404 returned error can't find the container with id 9cb54c9f5076a5f9b62359e4e678dee6d6dac3802e82cfa9f6a415162b3d2fd1 Sep 06 10:33:38.838000 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.837908 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" event={"ID":"4fcd0409-d67e-440a-8cf3-d438fe8e8cc9","Type":"ContainerStarted","Data":"c43ea3de8e08cdb6c3957bd17abb49895024140576507b07eeb2b6bb3a7ad6f6"} Sep 06 10:33:38.839444 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.839417 2574 generic.go:356] "Generic (PLEG): container finished" podID="d231825d-cbc1-4119-b092-a7faf0797b28" containerID="5f4f99a0d7b9d65a33f4d1237fc39bcabfdad31e24fc242eedb25aeb4466969b" exitCode=0 Sep 06 10:33:38.839543 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.839476 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" event={"ID":"d231825d-cbc1-4119-b092-a7faf0797b28","Type":"ContainerDied","Data":"5f4f99a0d7b9d65a33f4d1237fc39bcabfdad31e24fc242eedb25aeb4466969b"} Sep 06 10:33:38.839543 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:38.839507 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" event={"ID":"d231825d-cbc1-4119-b092-a7faf0797b28","Type":"ContainerStarted","Data":"9cb54c9f5076a5f9b62359e4e678dee6d6dac3802e82cfa9f6a415162b3d2fd1"} Sep 06 10:33:39.811521 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:39.811485 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:40.222517 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:40.222480 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:40.852335 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:40.852299 2574 generic.go:356] "Generic (PLEG): container finished" podID="d231825d-cbc1-4119-b092-a7faf0797b28" containerID="109c355d05df6ba475de44fddcdabeea5b5b734a0eddb734e47b6c400a7ca781" exitCode=0 Sep 06 10:33:40.852778 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:40.852347 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" event={"ID":"d231825d-cbc1-4119-b092-a7faf0797b28","Type":"ContainerDied","Data":"109c355d05df6ba475de44fddcdabeea5b5b734a0eddb734e47b6c400a7ca781"} Sep 06 10:33:41.862881 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:41.862839 2574 generic.go:356] "Generic (PLEG): container finished" podID="d231825d-cbc1-4119-b092-a7faf0797b28" containerID="49546221c635214702df3241043d6976ceb46c84e0ecba75224ef758c97c1d3a" exitCode=0 Sep 06 10:33:41.863375 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:41.862926 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" event={"ID":"d231825d-cbc1-4119-b092-a7faf0797b28","Type":"ContainerDied","Data":"49546221c635214702df3241043d6976ceb46c84e0ecba75224ef758c97c1d3a"} Sep 06 10:33:42.366673 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:42.366629 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:42.467253 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:42.467218 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:42.869254 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:42.869217 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" event={"ID":"4fcd0409-d67e-440a-8cf3-d438fe8e8cc9","Type":"ContainerStarted","Data":"ac6b33dfbaaa0f4a615a8457950d9c09f9316dda413857f759b343e5fdccb64d"} Sep 06 10:33:42.884440 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:42.884387 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" podStartSLOduration=1.660259585 podStartE2EDuration="5.884365769s" totalImagesPullingTime="4.224106184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:33:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:33:42.884252114 +0000 UTC m=+216.466741746" watchObservedRunningTime="2026-09-06 10:33:42.884365769 +0000 UTC m=+216.466855387" Sep 06 10:33:42.991435 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:42.991411 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:43.086341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.086311 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbr5n\" (UniqueName: \"kubernetes.io/projected/d231825d-cbc1-4119-b092-a7faf0797b28-kube-api-access-hbr5n\") pod \"d231825d-cbc1-4119-b092-a7faf0797b28\" (UID: \"d231825d-cbc1-4119-b092-a7faf0797b28\") " Sep 06 10:33:43.086487 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.086356 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d231825d-cbc1-4119-b092-a7faf0797b28-util\") pod \"d231825d-cbc1-4119-b092-a7faf0797b28\" (UID: \"d231825d-cbc1-4119-b092-a7faf0797b28\") " Sep 06 10:33:43.086487 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.086393 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d231825d-cbc1-4119-b092-a7faf0797b28-bundle\") pod \"d231825d-cbc1-4119-b092-a7faf0797b28\" (UID: \"d231825d-cbc1-4119-b092-a7faf0797b28\") " Sep 06 10:33:43.086810 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.086780 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d231825d-cbc1-4119-b092-a7faf0797b28-bundle" (OuterVolumeSpecName: "bundle") pod "d231825d-cbc1-4119-b092-a7faf0797b28" (UID: "d231825d-cbc1-4119-b092-a7faf0797b28"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:33:43.088221 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.088188 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d231825d-cbc1-4119-b092-a7faf0797b28-kube-api-access-hbr5n" (OuterVolumeSpecName: "kube-api-access-hbr5n") pod "d231825d-cbc1-4119-b092-a7faf0797b28" (UID: "d231825d-cbc1-4119-b092-a7faf0797b28"). InnerVolumeSpecName "kube-api-access-hbr5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:33:43.091508 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.091473 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d231825d-cbc1-4119-b092-a7faf0797b28-util" (OuterVolumeSpecName: "util") pod "d231825d-cbc1-4119-b092-a7faf0797b28" (UID: "d231825d-cbc1-4119-b092-a7faf0797b28"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:33:43.187167 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.187078 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbr5n\" (UniqueName: \"kubernetes.io/projected/d231825d-cbc1-4119-b092-a7faf0797b28-kube-api-access-hbr5n\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:33:43.187167 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.187103 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d231825d-cbc1-4119-b092-a7faf0797b28-util\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:33:43.187167 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.187113 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d231825d-cbc1-4119-b092-a7faf0797b28-bundle\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:33:43.876086 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.876053 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" Sep 06 10:33:43.876086 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.876081 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6wz5m" event={"ID":"d231825d-cbc1-4119-b092-a7faf0797b28","Type":"ContainerDied","Data":"9cb54c9f5076a5f9b62359e4e678dee6d6dac3802e82cfa9f6a415162b3d2fd1"} Sep 06 10:33:43.876525 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.876145 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9cb54c9f5076a5f9b62359e4e678dee6d6dac3802e82cfa9f6a415162b3d2fd1" Sep 06 10:33:43.876525 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:43.876331 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" Sep 06 10:33:47.069611 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:47.069577 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:47.169746 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:47.169708 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:48.016854 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:48.016811 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:48.976263 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:48.976231 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:49.079307 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.079277 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:49.351725 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.351643 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qxnfg"] Sep 06 10:33:49.351935 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.351923 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d231825d-cbc1-4119-b092-a7faf0797b28" containerName="extract" Sep 06 10:33:49.351982 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.351938 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d231825d-cbc1-4119-b092-a7faf0797b28" containerName="extract" Sep 06 10:33:49.351982 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.351953 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d231825d-cbc1-4119-b092-a7faf0797b28" containerName="pull" Sep 06 10:33:49.351982 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.351961 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d231825d-cbc1-4119-b092-a7faf0797b28" containerName="pull" Sep 06 10:33:49.351982 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.351971 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d231825d-cbc1-4119-b092-a7faf0797b28" containerName="util" Sep 06 10:33:49.351982 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.351976 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d231825d-cbc1-4119-b092-a7faf0797b28" containerName="util" Sep 06 10:33:49.352201 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.352031 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d231825d-cbc1-4119-b092-a7faf0797b28" containerName="extract" Sep 06 10:33:49.353859 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.353843 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qxnfg" Sep 06 10:33:49.356450 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.356429 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-sdndc\"" Sep 06 10:33:49.369251 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.369224 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qxnfg"] Sep 06 10:33:49.434286 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.434251 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftr9r\" (UniqueName: \"kubernetes.io/projected/2e990aeb-fea2-4555-a182-d05643d18808-kube-api-access-ftr9r\") pod \"cert-manager-55b4d954d5-qxnfg\" (UID: \"2e990aeb-fea2-4555-a182-d05643d18808\") " pod="cert-manager/cert-manager-55b4d954d5-qxnfg" Sep 06 10:33:49.434461 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.434320 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e990aeb-fea2-4555-a182-d05643d18808-bound-sa-token\") pod \"cert-manager-55b4d954d5-qxnfg\" (UID: \"2e990aeb-fea2-4555-a182-d05643d18808\") " pod="cert-manager/cert-manager-55b4d954d5-qxnfg" Sep 06 10:33:49.535603 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.535565 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e990aeb-fea2-4555-a182-d05643d18808-bound-sa-token\") pod \"cert-manager-55b4d954d5-qxnfg\" (UID: \"2e990aeb-fea2-4555-a182-d05643d18808\") " pod="cert-manager/cert-manager-55b4d954d5-qxnfg" Sep 06 10:33:49.535791 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.535621 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftr9r\" (UniqueName: \"kubernetes.io/projected/2e990aeb-fea2-4555-a182-d05643d18808-kube-api-access-ftr9r\") pod \"cert-manager-55b4d954d5-qxnfg\" (UID: \"2e990aeb-fea2-4555-a182-d05643d18808\") " pod="cert-manager/cert-manager-55b4d954d5-qxnfg" Sep 06 10:33:49.544082 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.544047 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e990aeb-fea2-4555-a182-d05643d18808-bound-sa-token\") pod \"cert-manager-55b4d954d5-qxnfg\" (UID: \"2e990aeb-fea2-4555-a182-d05643d18808\") " pod="cert-manager/cert-manager-55b4d954d5-qxnfg" Sep 06 10:33:49.544245 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.544163 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftr9r\" (UniqueName: \"kubernetes.io/projected/2e990aeb-fea2-4555-a182-d05643d18808-kube-api-access-ftr9r\") pod \"cert-manager-55b4d954d5-qxnfg\" (UID: \"2e990aeb-fea2-4555-a182-d05643d18808\") " pod="cert-manager/cert-manager-55b4d954d5-qxnfg" Sep 06 10:33:49.662270 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.662178 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qxnfg" Sep 06 10:33:49.882190 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.882158 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xkp5w" Sep 06 10:33:49.932259 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:49.932078 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qxnfg"] Sep 06 10:33:49.935432 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:33:49.935400 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e990aeb_fea2_4555_a182_d05643d18808.slice/crio-4a98c75fab2f9314681085d73933a69976e785a05bb8c931d5e6fe9f235ac9f3 WatchSource:0}: Error finding container 4a98c75fab2f9314681085d73933a69976e785a05bb8c931d5e6fe9f235ac9f3: Status 404 returned error can't find the container with id 4a98c75fab2f9314681085d73933a69976e785a05bb8c931d5e6fe9f235ac9f3 Sep 06 10:33:50.907375 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:50.907332 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qxnfg" event={"ID":"2e990aeb-fea2-4555-a182-d05643d18808","Type":"ContainerStarted","Data":"e942b94ef222ebfbb354df047a44c99ebe282ec3fe9df03b69771f6e82ca900b"} Sep 06 10:33:50.907737 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:50.907383 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qxnfg" event={"ID":"2e990aeb-fea2-4555-a182-d05643d18808","Type":"ContainerStarted","Data":"4a98c75fab2f9314681085d73933a69976e785a05bb8c931d5e6fe9f235ac9f3"} Sep 06 10:33:50.923840 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:50.923796 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-qxnfg" podStartSLOduration=1.923781722 podStartE2EDuration="1.923781722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:33:50.922082197 +0000 UTC m=+224.504571828" watchObservedRunningTime="2026-09-06 10:33:50.923781722 +0000 UTC m=+224.506271353" Sep 06 10:33:51.018020 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.017989 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk"] Sep 06 10:33:51.020069 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.020054 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk" Sep 06 10:33:51.022239 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.022215 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-crfbg\"" Sep 06 10:33:51.022328 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.022216 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 06 10:33:51.022328 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.022272 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 06 10:33:51.028828 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.028804 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk"] Sep 06 10:33:51.049162 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.049139 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdwlr\" (UniqueName: \"kubernetes.io/projected/00760b1e-0642-4080-b4a4-5b0484257ec4-kube-api-access-xdwlr\") pod \"openshift-lws-operator-fd8ccff4c-8w2qk\" (UID: \"00760b1e-0642-4080-b4a4-5b0484257ec4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk" Sep 06 10:33:51.049274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.049195 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00760b1e-0642-4080-b4a4-5b0484257ec4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8w2qk\" (UID: \"00760b1e-0642-4080-b4a4-5b0484257ec4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk" Sep 06 10:33:51.150560 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.150533 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdwlr\" (UniqueName: \"kubernetes.io/projected/00760b1e-0642-4080-b4a4-5b0484257ec4-kube-api-access-xdwlr\") pod \"openshift-lws-operator-fd8ccff4c-8w2qk\" (UID: \"00760b1e-0642-4080-b4a4-5b0484257ec4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk" Sep 06 10:33:51.150692 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.150573 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00760b1e-0642-4080-b4a4-5b0484257ec4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8w2qk\" (UID: \"00760b1e-0642-4080-b4a4-5b0484257ec4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk" Sep 06 10:33:51.150894 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.150878 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00760b1e-0642-4080-b4a4-5b0484257ec4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8w2qk\" (UID: \"00760b1e-0642-4080-b4a4-5b0484257ec4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk" Sep 06 10:33:51.162504 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.162449 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdwlr\" (UniqueName: \"kubernetes.io/projected/00760b1e-0642-4080-b4a4-5b0484257ec4-kube-api-access-xdwlr\") pod \"openshift-lws-operator-fd8ccff4c-8w2qk\" (UID: \"00760b1e-0642-4080-b4a4-5b0484257ec4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk" Sep 06 10:33:51.330431 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.330400 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk" Sep 06 10:33:51.598415 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.598383 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk"] Sep 06 10:33:51.912931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:51.912849 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk" event={"ID":"00760b1e-0642-4080-b4a4-5b0484257ec4","Type":"ContainerStarted","Data":"49dec3307de1967680be8ae8478b09d6767cf5285c1a52e8029f89aa4423915b"} Sep 06 10:33:54.461464 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:54.461427 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:54.926986 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:54.926957 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk" event={"ID":"00760b1e-0642-4080-b4a4-5b0484257ec4","Type":"ContainerStarted","Data":"65cf0ec9a787732c9a8d7c44819116060a3c8ef0f738e566c486fdeba1cbb44f"} Sep 06 10:33:54.942786 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:54.942737 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8w2qk" podStartSLOduration=1.016748195 podStartE2EDuration="3.942720982s" totalImagesPullingTime="2.925972787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:33:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:33:54.94173218 +0000 UTC m=+228.524221812" watchObservedRunningTime="2026-09-06 10:33:54.942720982 +0000 UTC m=+228.525210612" Sep 06 10:33:55.561679 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:55.561644 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:55.663103 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:55.663069 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:57.553342 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:57.553301 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:33:59.580987 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.580949 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v"] Sep 06 10:33:59.583258 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.583242 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:33:59.585711 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.585694 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 06 10:33:59.585838 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.585816 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 06 10:33:59.585934 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.585915 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jqnm5\"" Sep 06 10:33:59.592883 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.592863 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v"] Sep 06 10:33:59.715324 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.715289 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b480d90-ad3d-463b-b156-482cd07f0974-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v\" (UID: \"8b480d90-ad3d-463b-b156-482cd07f0974\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:33:59.715477 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.715338 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b480d90-ad3d-463b-b156-482cd07f0974-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v\" (UID: \"8b480d90-ad3d-463b-b156-482cd07f0974\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:33:59.715477 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.715417 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vm95\" (UniqueName: \"kubernetes.io/projected/8b480d90-ad3d-463b-b156-482cd07f0974-kube-api-access-7vm95\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v\" (UID: \"8b480d90-ad3d-463b-b156-482cd07f0974\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:33:59.816265 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.816226 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b480d90-ad3d-463b-b156-482cd07f0974-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v\" (UID: \"8b480d90-ad3d-463b-b156-482cd07f0974\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:33:59.816438 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.816280 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b480d90-ad3d-463b-b156-482cd07f0974-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v\" (UID: \"8b480d90-ad3d-463b-b156-482cd07f0974\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:33:59.816438 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.816336 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vm95\" (UniqueName: \"kubernetes.io/projected/8b480d90-ad3d-463b-b156-482cd07f0974-kube-api-access-7vm95\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v\" (UID: \"8b480d90-ad3d-463b-b156-482cd07f0974\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:33:59.816608 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.816588 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b480d90-ad3d-463b-b156-482cd07f0974-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v\" (UID: \"8b480d90-ad3d-463b-b156-482cd07f0974\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:33:59.816677 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.816620 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b480d90-ad3d-463b-b156-482cd07f0974-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v\" (UID: \"8b480d90-ad3d-463b-b156-482cd07f0974\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:33:59.825252 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.825232 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vm95\" (UniqueName: \"kubernetes.io/projected/8b480d90-ad3d-463b-b156-482cd07f0974-kube-api-access-7vm95\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v\" (UID: \"8b480d90-ad3d-463b-b156-482cd07f0974\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:33:59.891989 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:33:59.891917 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:34:00.160285 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:00.160259 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v"] Sep 06 10:34:00.162777 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:34:00.162749 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b480d90_ad3d_463b_b156_482cd07f0974.slice/crio-316849a61042b17f848acd8500aebaced35f2bd5a61f0dfbe02a2c3a2392e8cb WatchSource:0}: Error finding container 316849a61042b17f848acd8500aebaced35f2bd5a61f0dfbe02a2c3a2392e8cb: Status 404 returned error can't find the container with id 316849a61042b17f848acd8500aebaced35f2bd5a61f0dfbe02a2c3a2392e8cb Sep 06 10:34:00.953526 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:00.953485 2574 generic.go:356] "Generic (PLEG): container finished" podID="8b480d90-ad3d-463b-b156-482cd07f0974" containerID="642030c531df6651ccc440f7e77ffae239f5774beb1f145a98c5cc45179d492e" exitCode=0 Sep 06 10:34:00.953910 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:00.953574 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" event={"ID":"8b480d90-ad3d-463b-b156-482cd07f0974","Type":"ContainerDied","Data":"642030c531df6651ccc440f7e77ffae239f5774beb1f145a98c5cc45179d492e"} Sep 06 10:34:00.953910 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:00.953620 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" event={"ID":"8b480d90-ad3d-463b-b156-482cd07f0974","Type":"ContainerStarted","Data":"316849a61042b17f848acd8500aebaced35f2bd5a61f0dfbe02a2c3a2392e8cb"} Sep 06 10:34:01.959113 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:01.959082 2574 generic.go:356] "Generic (PLEG): container finished" podID="8b480d90-ad3d-463b-b156-482cd07f0974" containerID="7f631867a836c558d242dfc837292dffb3c1393be7596d03cc43708e1d7d4929" exitCode=0 Sep 06 10:34:01.959492 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:01.959168 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" event={"ID":"8b480d90-ad3d-463b-b156-482cd07f0974","Type":"ContainerDied","Data":"7f631867a836c558d242dfc837292dffb3c1393be7596d03cc43708e1d7d4929"} Sep 06 10:34:02.965373 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:02.965332 2574 generic.go:356] "Generic (PLEG): container finished" podID="8b480d90-ad3d-463b-b156-482cd07f0974" containerID="d64547f1d3f080b5712d6100f9deaed2e43e77bc394e328c329b3d489524f8f9" exitCode=0 Sep 06 10:34:02.965758 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:02.965420 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" event={"ID":"8b480d90-ad3d-463b-b156-482cd07f0974","Type":"ContainerDied","Data":"d64547f1d3f080b5712d6100f9deaed2e43e77bc394e328c329b3d489524f8f9"} Sep 06 10:34:04.099341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.099318 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:34:04.149817 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.149790 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b480d90-ad3d-463b-b156-482cd07f0974-util\") pod \"8b480d90-ad3d-463b-b156-482cd07f0974\" (UID: \"8b480d90-ad3d-463b-b156-482cd07f0974\") " Sep 06 10:34:04.149936 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.149859 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b480d90-ad3d-463b-b156-482cd07f0974-bundle\") pod \"8b480d90-ad3d-463b-b156-482cd07f0974\" (UID: \"8b480d90-ad3d-463b-b156-482cd07f0974\") " Sep 06 10:34:04.149936 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.149889 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vm95\" (UniqueName: \"kubernetes.io/projected/8b480d90-ad3d-463b-b156-482cd07f0974-kube-api-access-7vm95\") pod \"8b480d90-ad3d-463b-b156-482cd07f0974\" (UID: \"8b480d90-ad3d-463b-b156-482cd07f0974\") " Sep 06 10:34:04.150580 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.150552 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b480d90-ad3d-463b-b156-482cd07f0974-bundle" (OuterVolumeSpecName: "bundle") pod "8b480d90-ad3d-463b-b156-482cd07f0974" (UID: "8b480d90-ad3d-463b-b156-482cd07f0974"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:34:04.151952 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.151922 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b480d90-ad3d-463b-b156-482cd07f0974-kube-api-access-7vm95" (OuterVolumeSpecName: "kube-api-access-7vm95") pod "8b480d90-ad3d-463b-b156-482cd07f0974" (UID: "8b480d90-ad3d-463b-b156-482cd07f0974"). InnerVolumeSpecName "kube-api-access-7vm95". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:34:04.155541 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.155518 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b480d90-ad3d-463b-b156-482cd07f0974-util" (OuterVolumeSpecName: "util") pod "8b480d90-ad3d-463b-b156-482cd07f0974" (UID: "8b480d90-ad3d-463b-b156-482cd07f0974"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:34:04.250863 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.250796 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b480d90-ad3d-463b-b156-482cd07f0974-bundle\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:34:04.250863 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.250821 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vm95\" (UniqueName: \"kubernetes.io/projected/8b480d90-ad3d-463b-b156-482cd07f0974-kube-api-access-7vm95\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:34:04.250863 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.250831 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b480d90-ad3d-463b-b156-482cd07f0974-util\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:34:04.975840 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.975810 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" event={"ID":"8b480d90-ad3d-463b-b156-482cd07f0974","Type":"ContainerDied","Data":"316849a61042b17f848acd8500aebaced35f2bd5a61f0dfbe02a2c3a2392e8cb"} Sep 06 10:34:04.975975 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.975854 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="316849a61042b17f848acd8500aebaced35f2bd5a61f0dfbe02a2c3a2392e8cb" Sep 06 10:34:04.975975 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:04.975821 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5vt8v" Sep 06 10:34:08.803006 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.802969 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm"] Sep 06 10:34:08.803403 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.803274 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b480d90-ad3d-463b-b156-482cd07f0974" containerName="util" Sep 06 10:34:08.803403 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.803286 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b480d90-ad3d-463b-b156-482cd07f0974" containerName="util" Sep 06 10:34:08.803403 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.803299 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b480d90-ad3d-463b-b156-482cd07f0974" containerName="pull" Sep 06 10:34:08.803403 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.803304 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b480d90-ad3d-463b-b156-482cd07f0974" containerName="pull" Sep 06 10:34:08.803403 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.803322 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b480d90-ad3d-463b-b156-482cd07f0974" containerName="extract" Sep 06 10:34:08.803403 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.803327 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b480d90-ad3d-463b-b156-482cd07f0974" containerName="extract" Sep 06 10:34:08.803403 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.803381 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b480d90-ad3d-463b-b156-482cd07f0974" containerName="extract" Sep 06 10:34:08.806035 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.806014 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:08.808632 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.808604 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 06 10:34:08.808935 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.808613 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 06 10:34:08.809417 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.809379 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jqnm5\"" Sep 06 10:34:08.816305 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.816281 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm"] Sep 06 10:34:08.888871 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.888832 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74c16ea7-d564-469a-8828-fa7e065b5133-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm\" (UID: \"74c16ea7-d564-469a-8828-fa7e065b5133\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:08.889063 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.888879 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc4bw\" (UniqueName: \"kubernetes.io/projected/74c16ea7-d564-469a-8828-fa7e065b5133-kube-api-access-zc4bw\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm\" (UID: \"74c16ea7-d564-469a-8828-fa7e065b5133\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:08.889063 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.888910 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74c16ea7-d564-469a-8828-fa7e065b5133-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm\" (UID: \"74c16ea7-d564-469a-8828-fa7e065b5133\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:08.989691 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.989652 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74c16ea7-d564-469a-8828-fa7e065b5133-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm\" (UID: \"74c16ea7-d564-469a-8828-fa7e065b5133\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:08.989691 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.989690 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc4bw\" (UniqueName: \"kubernetes.io/projected/74c16ea7-d564-469a-8828-fa7e065b5133-kube-api-access-zc4bw\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm\" (UID: \"74c16ea7-d564-469a-8828-fa7e065b5133\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:08.989903 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.989710 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74c16ea7-d564-469a-8828-fa7e065b5133-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm\" (UID: \"74c16ea7-d564-469a-8828-fa7e065b5133\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:08.990058 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.990033 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74c16ea7-d564-469a-8828-fa7e065b5133-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm\" (UID: \"74c16ea7-d564-469a-8828-fa7e065b5133\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:08.990205 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.990080 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74c16ea7-d564-469a-8828-fa7e065b5133-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm\" (UID: \"74c16ea7-d564-469a-8828-fa7e065b5133\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:08.997930 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:08.997902 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc4bw\" (UniqueName: \"kubernetes.io/projected/74c16ea7-d564-469a-8828-fa7e065b5133-kube-api-access-zc4bw\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm\" (UID: \"74c16ea7-d564-469a-8828-fa7e065b5133\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:09.116343 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:09.116241 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:09.395460 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:09.395397 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm"] Sep 06 10:34:09.397460 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:34:09.397431 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74c16ea7_d564_469a_8828_fa7e065b5133.slice/crio-49abb3717eb0f7a6fd11917b03f859aadab4d797e7133badbb8e1fd4af45b107 WatchSource:0}: Error finding container 49abb3717eb0f7a6fd11917b03f859aadab4d797e7133badbb8e1fd4af45b107: Status 404 returned error can't find the container with id 49abb3717eb0f7a6fd11917b03f859aadab4d797e7133badbb8e1fd4af45b107 Sep 06 10:34:09.893734 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:09.893688 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:09.992170 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:09.992115 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:09.997431 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:09.997406 2574 generic.go:356] "Generic (PLEG): container finished" podID="74c16ea7-d564-469a-8828-fa7e065b5133" containerID="1e0d209ca6192643574a2511b27132f9a6913055075eeb0fdbec35cac9575022" exitCode=0 Sep 06 10:34:09.997570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:09.997479 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" event={"ID":"74c16ea7-d564-469a-8828-fa7e065b5133","Type":"ContainerDied","Data":"1e0d209ca6192643574a2511b27132f9a6913055075eeb0fdbec35cac9575022"} Sep 06 10:34:09.997570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:09.997526 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" event={"ID":"74c16ea7-d564-469a-8828-fa7e065b5133","Type":"ContainerStarted","Data":"49abb3717eb0f7a6fd11917b03f859aadab4d797e7133badbb8e1fd4af45b107"} Sep 06 10:34:10.522142 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:10.522087 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:10.720841 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:10.720813 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:10.832944 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:10.832867 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:10.856271 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:10.856240 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs"] Sep 06 10:34:10.858300 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:10.858287 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:10.861448 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:10.861431 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 06 10:34:10.861561 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:10.861541 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 06 10:34:10.861719 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:10.861702 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fg8xs\"" Sep 06 10:34:10.861790 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:10.861738 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 06 10:34:10.861898 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:10.861879 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 06 10:34:10.875550 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:10.875528 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs"] Sep 06 10:34:11.003668 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.003631 2574 generic.go:356] "Generic (PLEG): container finished" podID="74c16ea7-d564-469a-8828-fa7e065b5133" containerID="9e277f7087a10e47d91131cdc20ec12686cdbdcc33c391176f33ffb5cd25d3b5" exitCode=0 Sep 06 10:34:11.004076 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.003725 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" event={"ID":"74c16ea7-d564-469a-8828-fa7e065b5133","Type":"ContainerDied","Data":"9e277f7087a10e47d91131cdc20ec12686cdbdcc33c391176f33ffb5cd25d3b5"} Sep 06 10:34:11.004076 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.003825 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d45ffaa-1462-4ec1-a397-9cca1f483cce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.004076 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.003888 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d45ffaa-1462-4ec1-a397-9cca1f483cce-webhook-cert\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.004076 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.003963 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d45ffaa-1462-4ec1-a397-9cca1f483cce-tmp\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.004289 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.004094 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d45ffaa-1462-4ec1-a397-9cca1f483cce-manifests\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.004327 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.004308 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7rf2\" (UniqueName: \"kubernetes.io/projected/1d45ffaa-1462-4ec1-a397-9cca1f483cce-kube-api-access-b7rf2\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.105553 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.105465 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d45ffaa-1462-4ec1-a397-9cca1f483cce-manifests\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.105553 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.105511 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7rf2\" (UniqueName: \"kubernetes.io/projected/1d45ffaa-1462-4ec1-a397-9cca1f483cce-kube-api-access-b7rf2\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.105553 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.105549 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d45ffaa-1462-4ec1-a397-9cca1f483cce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.105769 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.105600 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d45ffaa-1462-4ec1-a397-9cca1f483cce-webhook-cert\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.105769 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.105636 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d45ffaa-1462-4ec1-a397-9cca1f483cce-tmp\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.105962 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.105929 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d45ffaa-1462-4ec1-a397-9cca1f483cce-manifests\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.107945 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.107919 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d45ffaa-1462-4ec1-a397-9cca1f483cce-tmp\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.108421 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.108399 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d45ffaa-1462-4ec1-a397-9cca1f483cce-apiservice-cert\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.108537 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.108521 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d45ffaa-1462-4ec1-a397-9cca1f483cce-webhook-cert\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.114928 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.114903 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7rf2\" (UniqueName: \"kubernetes.io/projected/1d45ffaa-1462-4ec1-a397-9cca1f483cce-kube-api-access-b7rf2\") pod \"opendatahub-operator-controller-manager-645f664568-2zfbs\" (UID: \"1d45ffaa-1462-4ec1-a397-9cca1f483cce\") " pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.125212 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.125186 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:11.167807 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.167781 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:11.224970 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.224937 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:11.323394 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.323352 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:11.427367 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.427321 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:11.442377 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.442349 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs"] Sep 06 10:34:11.445437 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:34:11.445412 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d45ffaa_1462_4ec1_a397_9cca1f483cce.slice/crio-0b0efe6e688456b769d2d15041ea2865dd215d0d81b6d7f0053dba344dfb8544 WatchSource:0}: Error finding container 0b0efe6e688456b769d2d15041ea2865dd215d0d81b6d7f0053dba344dfb8544: Status 404 returned error can't find the container with id 0b0efe6e688456b769d2d15041ea2865dd215d0d81b6d7f0053dba344dfb8544 Sep 06 10:34:11.525257 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.525224 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:11.624109 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.624078 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:11.725754 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:11.725681 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:12.013646 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:12.013557 2574 generic.go:356] "Generic (PLEG): container finished" podID="74c16ea7-d564-469a-8828-fa7e065b5133" containerID="b9949611f4b2917e944db95a76cac799c387a8f76704502c3517315f26019aa7" exitCode=0 Sep 06 10:34:12.014082 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:12.013664 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" event={"ID":"74c16ea7-d564-469a-8828-fa7e065b5133","Type":"ContainerDied","Data":"b9949611f4b2917e944db95a76cac799c387a8f76704502c3517315f26019aa7"} Sep 06 10:34:12.015803 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:12.015767 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" event={"ID":"1d45ffaa-1462-4ec1-a397-9cca1f483cce","Type":"ContainerStarted","Data":"0b0efe6e688456b769d2d15041ea2865dd215d0d81b6d7f0053dba344dfb8544"} Sep 06 10:34:14.273908 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:14.273882 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:14.335770 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:14.335749 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74c16ea7-d564-469a-8828-fa7e065b5133-util\") pod \"74c16ea7-d564-469a-8828-fa7e065b5133\" (UID: \"74c16ea7-d564-469a-8828-fa7e065b5133\") " Sep 06 10:34:14.335883 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:14.335801 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zc4bw\" (UniqueName: \"kubernetes.io/projected/74c16ea7-d564-469a-8828-fa7e065b5133-kube-api-access-zc4bw\") pod \"74c16ea7-d564-469a-8828-fa7e065b5133\" (UID: \"74c16ea7-d564-469a-8828-fa7e065b5133\") " Sep 06 10:34:14.335883 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:14.335826 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74c16ea7-d564-469a-8828-fa7e065b5133-bundle\") pod \"74c16ea7-d564-469a-8828-fa7e065b5133\" (UID: \"74c16ea7-d564-469a-8828-fa7e065b5133\") " Sep 06 10:34:14.336650 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:14.336616 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74c16ea7-d564-469a-8828-fa7e065b5133-bundle" (OuterVolumeSpecName: "bundle") pod "74c16ea7-d564-469a-8828-fa7e065b5133" (UID: "74c16ea7-d564-469a-8828-fa7e065b5133"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:34:14.337683 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:14.337662 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74c16ea7-d564-469a-8828-fa7e065b5133-kube-api-access-zc4bw" (OuterVolumeSpecName: "kube-api-access-zc4bw") pod "74c16ea7-d564-469a-8828-fa7e065b5133" (UID: "74c16ea7-d564-469a-8828-fa7e065b5133"). InnerVolumeSpecName "kube-api-access-zc4bw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:34:14.341018 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:14.340996 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74c16ea7-d564-469a-8828-fa7e065b5133-util" (OuterVolumeSpecName: "util") pod "74c16ea7-d564-469a-8828-fa7e065b5133" (UID: "74c16ea7-d564-469a-8828-fa7e065b5133"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:34:14.437225 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:14.437194 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc4bw\" (UniqueName: \"kubernetes.io/projected/74c16ea7-d564-469a-8828-fa7e065b5133-kube-api-access-zc4bw\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:34:14.437225 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:14.437222 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74c16ea7-d564-469a-8828-fa7e065b5133-bundle\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:34:14.437368 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:14.437234 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74c16ea7-d564-469a-8828-fa7e065b5133-util\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:34:15.031280 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:15.031242 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" event={"ID":"1d45ffaa-1462-4ec1-a397-9cca1f483cce","Type":"ContainerStarted","Data":"9b45021329df3ce3ce4a3c355340285a37cf4306095551a192bd546321f57d3b"} Sep 06 10:34:15.033355 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:15.033330 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" event={"ID":"74c16ea7-d564-469a-8828-fa7e065b5133","Type":"ContainerDied","Data":"49abb3717eb0f7a6fd11917b03f859aadab4d797e7133badbb8e1fd4af45b107"} Sep 06 10:34:15.033355 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:15.033347 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0btbwm" Sep 06 10:34:15.033494 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:15.033373 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49abb3717eb0f7a6fd11917b03f859aadab4d797e7133badbb8e1fd4af45b107" Sep 06 10:34:16.039716 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:16.039683 2574 generic.go:356] "Generic (PLEG): container finished" podID="1d45ffaa-1462-4ec1-a397-9cca1f483cce" containerID="9b45021329df3ce3ce4a3c355340285a37cf4306095551a192bd546321f57d3b" exitCode=0 Sep 06 10:34:16.040171 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:16.039737 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" event={"ID":"1d45ffaa-1462-4ec1-a397-9cca1f483cce","Type":"ContainerDied","Data":"9b45021329df3ce3ce4a3c355340285a37cf4306095551a192bd546321f57d3b"} Sep 06 10:34:17.046007 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:17.045969 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" event={"ID":"1d45ffaa-1462-4ec1-a397-9cca1f483cce","Type":"ContainerStarted","Data":"46716fc1bb0d8ce31e6683cbc721ec072d2e9644de2d67ab55c6f57586ae6e97"} Sep 06 10:34:17.046544 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:17.046105 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:17.067270 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:17.067218 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" podStartSLOduration=2.790917794 podStartE2EDuration="7.067201003s" totalImagesPullingTime="3.21268163s" totalInitContainerRuntime="1.063601579s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:34:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:34:17.066236713 +0000 UTC m=+250.648726343" watchObservedRunningTime="2026-09-06 10:34:17.067201003 +0000 UTC m=+250.649690626" Sep 06 10:34:21.620742 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:21.620706 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:21.723315 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:21.723276 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:21.833847 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:21.833817 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:21.935363 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:21.935326 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:28.052500 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:28.052467 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-645f664568-2zfbs" Sep 06 10:34:28.976185 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:28.976152 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:29.911920 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:29.911886 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:30.018778 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.018744 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:30.440885 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.440853 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6"] Sep 06 10:34:30.441164 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.441153 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74c16ea7-d564-469a-8828-fa7e065b5133" containerName="extract" Sep 06 10:34:30.441213 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.441166 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="74c16ea7-d564-469a-8828-fa7e065b5133" containerName="extract" Sep 06 10:34:30.441213 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.441182 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74c16ea7-d564-469a-8828-fa7e065b5133" containerName="util" Sep 06 10:34:30.441213 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.441187 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="74c16ea7-d564-469a-8828-fa7e065b5133" containerName="util" Sep 06 10:34:30.441213 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.441202 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74c16ea7-d564-469a-8828-fa7e065b5133" containerName="pull" Sep 06 10:34:30.441213 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.441208 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="74c16ea7-d564-469a-8828-fa7e065b5133" containerName="pull" Sep 06 10:34:30.441361 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.441259 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="74c16ea7-d564-469a-8828-fa7e065b5133" containerName="extract" Sep 06 10:34:30.444508 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.444477 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.449013 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.448989 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 06 10:34:30.449578 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.449564 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 06 10:34:30.449759 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.449741 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-p6lgl\"" Sep 06 10:34:30.449830 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.449756 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 06 10:34:30.455913 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.455889 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6"] Sep 06 10:34:30.570932 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.570901 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b1c0066-771d-4f54-9d63-60718e979073-metrics-cert\") pod \"lws-controller-manager-59b7bf7857-tmvq6\" (UID: \"8b1c0066-771d-4f54-9d63-60718e979073\") " pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.571110 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.570963 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b1c0066-771d-4f54-9d63-60718e979073-cert\") pod \"lws-controller-manager-59b7bf7857-tmvq6\" (UID: \"8b1c0066-771d-4f54-9d63-60718e979073\") " pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.571110 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.571041 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ldtg\" (UniqueName: \"kubernetes.io/projected/8b1c0066-771d-4f54-9d63-60718e979073-kube-api-access-7ldtg\") pod \"lws-controller-manager-59b7bf7857-tmvq6\" (UID: \"8b1c0066-771d-4f54-9d63-60718e979073\") " pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.571110 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.571091 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b1c0066-771d-4f54-9d63-60718e979073-manager-config\") pod \"lws-controller-manager-59b7bf7857-tmvq6\" (UID: \"8b1c0066-771d-4f54-9d63-60718e979073\") " pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.671981 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.671945 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ldtg\" (UniqueName: \"kubernetes.io/projected/8b1c0066-771d-4f54-9d63-60718e979073-kube-api-access-7ldtg\") pod \"lws-controller-manager-59b7bf7857-tmvq6\" (UID: \"8b1c0066-771d-4f54-9d63-60718e979073\") " pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.672181 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.671993 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b1c0066-771d-4f54-9d63-60718e979073-manager-config\") pod \"lws-controller-manager-59b7bf7857-tmvq6\" (UID: \"8b1c0066-771d-4f54-9d63-60718e979073\") " pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.672181 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.672013 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b1c0066-771d-4f54-9d63-60718e979073-metrics-cert\") pod \"lws-controller-manager-59b7bf7857-tmvq6\" (UID: \"8b1c0066-771d-4f54-9d63-60718e979073\") " pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.672181 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.672063 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b1c0066-771d-4f54-9d63-60718e979073-cert\") pod \"lws-controller-manager-59b7bf7857-tmvq6\" (UID: \"8b1c0066-771d-4f54-9d63-60718e979073\") " pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.672780 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.672753 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b1c0066-771d-4f54-9d63-60718e979073-manager-config\") pod \"lws-controller-manager-59b7bf7857-tmvq6\" (UID: \"8b1c0066-771d-4f54-9d63-60718e979073\") " pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.674716 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.674692 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b1c0066-771d-4f54-9d63-60718e979073-cert\") pod \"lws-controller-manager-59b7bf7857-tmvq6\" (UID: \"8b1c0066-771d-4f54-9d63-60718e979073\") " pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.674716 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.674713 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b1c0066-771d-4f54-9d63-60718e979073-metrics-cert\") pod \"lws-controller-manager-59b7bf7857-tmvq6\" (UID: \"8b1c0066-771d-4f54-9d63-60718e979073\") " pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.681161 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.681120 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ldtg\" (UniqueName: \"kubernetes.io/projected/8b1c0066-771d-4f54-9d63-60718e979073-kube-api-access-7ldtg\") pod \"lws-controller-manager-59b7bf7857-tmvq6\" (UID: \"8b1c0066-771d-4f54-9d63-60718e979073\") " pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:30.754998 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:30.754923 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:31.024981 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:31.024957 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6"] Sep 06 10:34:31.027577 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:34:31.027550 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b1c0066_771d_4f54_9d63_60718e979073.slice/crio-7f6dce1d800df8e4b3c191e749aa1f2a9ecb291dd326e34b405137b49b4ec53e WatchSource:0}: Error finding container 7f6dce1d800df8e4b3c191e749aa1f2a9ecb291dd326e34b405137b49b4ec53e: Status 404 returned error can't find the container with id 7f6dce1d800df8e4b3c191e749aa1f2a9ecb291dd326e34b405137b49b4ec53e Sep 06 10:34:31.108815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:31.108779 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" event={"ID":"8b1c0066-771d-4f54-9d63-60718e979073","Type":"ContainerStarted","Data":"7f6dce1d800df8e4b3c191e749aa1f2a9ecb291dd326e34b405137b49b4ec53e"} Sep 06 10:34:33.120849 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:33.120814 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" event={"ID":"8b1c0066-771d-4f54-9d63-60718e979073","Type":"ContainerStarted","Data":"bcc46c852cdb74bbd781de9c27764de265e16a3b1e73285d9a58253452042c7d"} Sep 06 10:34:33.121248 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:33.120957 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:33.137500 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:33.137453 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" podStartSLOduration=1.6116750720000002 podStartE2EDuration="3.137438586s" totalImagesPullingTime="1.525763514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:34:33.136378233 +0000 UTC m=+266.718867863" watchObservedRunningTime="2026-09-06 10:34:33.137438586 +0000 UTC m=+266.719928217" Sep 06 10:34:35.447779 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:35.447743 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:35.727673 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:35.727589 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:36.683889 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:36.683857 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:36.744372 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:36.744341 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:37.640700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:37.640670 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:40.918371 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:40.918330 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tvq92"] Sep 06 10:34:40.923440 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:40.923410 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" Sep 06 10:34:40.925931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:40.925902 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 06 10:34:40.926586 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:40.926562 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zv5bw\"" Sep 06 10:34:40.937517 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:40.937496 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tvq92"] Sep 06 10:34:40.946294 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:40.946257 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:40.964736 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:40.964709 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2-cert\") pod \"odh-model-controller-7998c67dc4-tvq92\" (UID: \"7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" Sep 06 10:34:40.964899 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:40.964771 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmpcc\" (UniqueName: \"kubernetes.io/projected/7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2-kube-api-access-kmpcc\") pod \"odh-model-controller-7998c67dc4-tvq92\" (UID: \"7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" Sep 06 10:34:41.044901 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.044845 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:41.065860 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.065828 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2-cert\") pod \"odh-model-controller-7998c67dc4-tvq92\" (UID: \"7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" Sep 06 10:34:41.066015 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.065874 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmpcc\" (UniqueName: \"kubernetes.io/projected/7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2-kube-api-access-kmpcc\") pod \"odh-model-controller-7998c67dc4-tvq92\" (UID: \"7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" Sep 06 10:34:41.066015 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:34:41.065980 2574 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 06 10:34:41.066096 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:34:41.066051 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2-cert podName:7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2 nodeName:}" failed. No retries permitted until 2026-09-06 10:34:41.566033798 +0000 UTC m=+275.148523411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2-cert") pod "odh-model-controller-7998c67dc4-tvq92" (UID: "7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2") : secret "odh-model-controller-webhook-cert" not found Sep 06 10:34:41.074378 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.074353 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmpcc\" (UniqueName: \"kubernetes.io/projected/7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2-kube-api-access-kmpcc\") pod \"odh-model-controller-7998c67dc4-tvq92\" (UID: \"7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" Sep 06 10:34:41.144499 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.144445 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:41.410679 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.410602 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:41.511332 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.511294 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:41.527581 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.527554 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zn5wj"] Sep 06 10:34:41.530866 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.530849 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" Sep 06 10:34:41.533400 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.533364 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 06 10:34:41.533732 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.533714 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-5jfx4\"" Sep 06 10:34:41.542457 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.542433 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zn5wj"] Sep 06 10:34:41.569315 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.569282 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9q9t\" (UniqueName: \"kubernetes.io/projected/ea239aef-7690-41a3-8a78-5e216efc0b45-kube-api-access-w9q9t\") pod \"model-serving-api-54d89bd5fc-zn5wj\" (UID: \"ea239aef-7690-41a3-8a78-5e216efc0b45\") " pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" Sep 06 10:34:41.569460 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.569331 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2-cert\") pod \"odh-model-controller-7998c67dc4-tvq92\" (UID: \"7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" Sep 06 10:34:41.569460 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.569358 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea239aef-7690-41a3-8a78-5e216efc0b45-tls-certs\") pod \"model-serving-api-54d89bd5fc-zn5wj\" (UID: \"ea239aef-7690-41a3-8a78-5e216efc0b45\") " pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" Sep 06 10:34:41.569460 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:34:41.569439 2574 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 06 10:34:41.569572 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:34:41.569492 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2-cert podName:7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2 nodeName:}" failed. No retries permitted until 2026-09-06 10:34:42.569477555 +0000 UTC m=+276.151967164 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2-cert") pod "odh-model-controller-7998c67dc4-tvq92" (UID: "7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2") : secret "odh-model-controller-webhook-cert" not found Sep 06 10:34:41.612890 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.612856 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:41.670328 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.670305 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9q9t\" (UniqueName: \"kubernetes.io/projected/ea239aef-7690-41a3-8a78-5e216efc0b45-kube-api-access-w9q9t\") pod \"model-serving-api-54d89bd5fc-zn5wj\" (UID: \"ea239aef-7690-41a3-8a78-5e216efc0b45\") " pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" Sep 06 10:34:41.670472 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.670360 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea239aef-7690-41a3-8a78-5e216efc0b45-tls-certs\") pod \"model-serving-api-54d89bd5fc-zn5wj\" (UID: \"ea239aef-7690-41a3-8a78-5e216efc0b45\") " pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" Sep 06 10:34:41.672752 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.672722 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea239aef-7690-41a3-8a78-5e216efc0b45-tls-certs\") pod \"model-serving-api-54d89bd5fc-zn5wj\" (UID: \"ea239aef-7690-41a3-8a78-5e216efc0b45\") " pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" Sep 06 10:34:41.678099 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.678060 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9q9t\" (UniqueName: \"kubernetes.io/projected/ea239aef-7690-41a3-8a78-5e216efc0b45-kube-api-access-w9q9t\") pod \"model-serving-api-54d89bd5fc-zn5wj\" (UID: \"ea239aef-7690-41a3-8a78-5e216efc0b45\") " pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" Sep 06 10:34:41.709522 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.709491 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:41.812049 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.812016 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:41.843144 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:41.843091 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" Sep 06 10:34:42.113779 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.113750 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zn5wj"] Sep 06 10:34:42.116073 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:34:42.116048 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea239aef_7690_41a3_8a78_5e216efc0b45.slice/crio-791d7953b883eb7981451974d4ed7c7c0116cd8422f9960820b19ebe81fbfe2a WatchSource:0}: Error finding container 791d7953b883eb7981451974d4ed7c7c0116cd8422f9960820b19ebe81fbfe2a: Status 404 returned error can't find the container with id 791d7953b883eb7981451974d4ed7c7c0116cd8422f9960820b19ebe81fbfe2a Sep 06 10:34:42.164150 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.164105 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" event={"ID":"ea239aef-7690-41a3-8a78-5e216efc0b45","Type":"ContainerStarted","Data":"791d7953b883eb7981451974d4ed7c7c0116cd8422f9960820b19ebe81fbfe2a"} Sep 06 10:34:42.169241 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.169212 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk"] Sep 06 10:34:42.174232 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.174207 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.176881 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.176712 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 06 10:34:42.176881 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.176716 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 06 10:34:42.176881 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.176722 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 06 10:34:42.177176 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.176995 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-czcj2\"" Sep 06 10:34:42.183711 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.183687 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk"] Sep 06 10:34:42.209929 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.209899 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:42.276440 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.276406 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.276440 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.276443 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.276689 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.276566 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.276689 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.276646 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.276689 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.276678 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.276851 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.276747 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.276851 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.276787 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmx25\" (UniqueName: \"kubernetes.io/projected/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-kube-api-access-fmx25\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.276851 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.276820 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.276851 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.276845 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.377648 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.377558 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.377648 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.377632 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.377872 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.377667 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.377872 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.377731 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.377985 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.377893 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmx25\" (UniqueName: \"kubernetes.io/projected/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-kube-api-access-fmx25\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.377985 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.377942 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.377985 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.377970 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.378200 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.378015 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.378200 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.378045 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.378200 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.378043 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.378856 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.378829 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.378963 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.378902 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.379176 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.379149 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.379262 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.379198 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.380265 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.380243 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.381322 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.381299 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.386320 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.386297 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.386703 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.386681 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmx25\" (UniqueName: \"kubernetes.io/projected/2ce36125-a847-43b9-bab2-29bd4bf0f8f5-kube-api-access-fmx25\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk\" (UID: \"2ce36125-a847-43b9-bab2-29bd4bf0f8f5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.466289 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.466229 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:42.487511 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.487477 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:42.579652 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.579610 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2-cert\") pod \"odh-model-controller-7998c67dc4-tvq92\" (UID: \"7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" Sep 06 10:34:42.582141 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.582087 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2-cert\") pod \"odh-model-controller-7998c67dc4-tvq92\" (UID: \"7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" Sep 06 10:34:42.592808 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.592784 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:42.736225 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.736189 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" Sep 06 10:34:42.776616 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:42.776566 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk"] Sep 06 10:34:43.034188 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:43.034154 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tvq92"] Sep 06 10:34:43.036022 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:34:43.035984 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d28b3ec_4bec_4f03_a4d2_c724aedaf2b2.slice/crio-42bf01169bcdc624a6e4bf97f4a82a54ef85002be8a75f5a5880bf0a8c4fe471 WatchSource:0}: Error finding container 42bf01169bcdc624a6e4bf97f4a82a54ef85002be8a75f5a5880bf0a8c4fe471: Status 404 returned error can't find the container with id 42bf01169bcdc624a6e4bf97f4a82a54ef85002be8a75f5a5880bf0a8c4fe471 Sep 06 10:34:43.173980 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:43.173945 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" event={"ID":"7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2","Type":"ContainerStarted","Data":"42bf01169bcdc624a6e4bf97f4a82a54ef85002be8a75f5a5880bf0a8c4fe471"} Sep 06 10:34:43.175833 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:43.175792 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" event={"ID":"2ce36125-a847-43b9-bab2-29bd4bf0f8f5","Type":"ContainerStarted","Data":"f67cb225bc962caf71c6f0bc47988c166f63374399ffb03b28e7da93aa484b57"} Sep 06 10:34:44.127614 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:44.127581 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59b7bf7857-tmvq6" Sep 06 10:34:45.195278 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:45.195234 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" event={"ID":"ea239aef-7690-41a3-8a78-5e216efc0b45","Type":"ContainerStarted","Data":"bfcaf08d36b17c543e898a513922f509d0ad37807842d3dc10cfb7dd1cfdc77f"} Sep 06 10:34:45.196067 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:45.196047 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" Sep 06 10:34:48.067185 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:48.067064 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:34:48.067185 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:48.067166 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:34:48.067490 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:48.067197 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:34:48.221482 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:48.221183 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" event={"ID":"7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2","Type":"ContainerStarted","Data":"e5d961cc3e1e4dc19ce287fd8844b6a48b77f84772bfd859cb6010e67b9c91d9"} Sep 06 10:34:48.221482 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:48.221456 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" Sep 06 10:34:48.223860 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:48.223831 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" event={"ID":"2ce36125-a847-43b9-bab2-29bd4bf0f8f5","Type":"ContainerStarted","Data":"c7e42d59f10611fa07aed2a2be2bfc5ad99025ba8ddfbc28d73fc1297fa4432a"} Sep 06 10:34:48.253995 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:48.253940 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" podStartSLOduration=4.671542137 podStartE2EDuration="7.253923797s" totalImagesPullingTime="2.58238166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:34:45.214170137 +0000 UTC m=+278.796659775" watchObservedRunningTime="2026-09-06 10:34:48.253923797 +0000 UTC m=+281.836413477" Sep 06 10:34:48.321412 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:48.321318 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" podStartSLOduration=3.292471023 podStartE2EDuration="8.321302283s" totalImagesPullingTime="5.02883126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:34:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:34:48.255264328 +0000 UTC m=+281.837753960" watchObservedRunningTime="2026-09-06 10:34:48.321302283 +0000 UTC m=+281.903791952" Sep 06 10:34:48.321412 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:48.321398 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" podStartSLOduration=1.0400564939999999 podStartE2EDuration="6.3213923s" totalImagesPullingTime="5.281335806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:34:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:34:48.318912839 +0000 UTC m=+281.901402471" watchObservedRunningTime="2026-09-06 10:34:48.3213923 +0000 UTC m=+281.903881930" Sep 06 10:34:48.488165 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:48.488114 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:49.015512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.015474 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd"] Sep 06 10:34:49.018944 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.018923 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.021081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.021057 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 06 10:34:49.021209 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.021060 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 06 10:34:49.021209 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.021193 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dv29f\"" Sep 06 10:34:49.021321 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.021226 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 06 10:34:49.029907 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.029881 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd"] Sep 06 10:34:49.143717 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.143676 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0719fc1d-2dea-450e-b161-8e2361629cf0-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.144150 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.143722 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0719fc1d-2dea-450e-b161-8e2361629cf0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.144150 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.143803 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0719fc1d-2dea-450e-b161-8e2361629cf0-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.144150 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.143828 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0719fc1d-2dea-450e-b161-8e2361629cf0-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.144150 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.143936 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0719fc1d-2dea-450e-b161-8e2361629cf0-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.144150 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.143973 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0719fc1d-2dea-450e-b161-8e2361629cf0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.144150 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.144002 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbczb\" (UniqueName: \"kubernetes.io/projected/0719fc1d-2dea-450e-b161-8e2361629cf0-kube-api-access-hbczb\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.244987 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.244952 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0719fc1d-2dea-450e-b161-8e2361629cf0-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.244987 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.244986 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0719fc1d-2dea-450e-b161-8e2361629cf0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.245210 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.245018 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0719fc1d-2dea-450e-b161-8e2361629cf0-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.245256 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.245239 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0719fc1d-2dea-450e-b161-8e2361629cf0-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.245318 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.245304 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0719fc1d-2dea-450e-b161-8e2361629cf0-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.245380 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.245325 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0719fc1d-2dea-450e-b161-8e2361629cf0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.245380 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.245342 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbczb\" (UniqueName: \"kubernetes.io/projected/0719fc1d-2dea-450e-b161-8e2361629cf0-kube-api-access-hbczb\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.245987 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.245964 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0719fc1d-2dea-450e-b161-8e2361629cf0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.247519 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.247489 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0719fc1d-2dea-450e-b161-8e2361629cf0-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.247610 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.247573 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0719fc1d-2dea-450e-b161-8e2361629cf0-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.247722 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.247707 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0719fc1d-2dea-450e-b161-8e2361629cf0-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.247762 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.247745 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0719fc1d-2dea-450e-b161-8e2361629cf0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.253458 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.253439 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0719fc1d-2dea-450e-b161-8e2361629cf0-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.253754 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.253737 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbczb\" (UniqueName: \"kubernetes.io/projected/0719fc1d-2dea-450e-b161-8e2361629cf0-kube-api-access-hbczb\") pod \"istiod-openshift-gateway-5677ccb49b-m7ftd\" (UID: \"0719fc1d-2dea-450e-b161-8e2361629cf0\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.328738 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.328660 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:49.492535 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.492502 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:49.605508 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:49.605435 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd"] Sep 06 10:34:49.609435 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:34:49.609403 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0719fc1d_2dea_450e_b161_8e2361629cf0.slice/crio-625ca09dac341d3323fc23c406629911e55ec6cde844a640f2e90a21b65c1224 WatchSource:0}: Error finding container 625ca09dac341d3323fc23c406629911e55ec6cde844a640f2e90a21b65c1224: Status 404 returned error can't find the container with id 625ca09dac341d3323fc23c406629911e55ec6cde844a640f2e90a21b65c1224 Sep 06 10:34:50.236249 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:50.236206 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" event={"ID":"0719fc1d-2dea-450e-b161-8e2361629cf0","Type":"ContainerStarted","Data":"625ca09dac341d3323fc23c406629911e55ec6cde844a640f2e90a21b65c1224"} Sep 06 10:34:50.236673 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:50.236540 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:50.238010 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:50.237987 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk" Sep 06 10:34:51.877323 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:51.877286 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:34:51.877582 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:51.877357 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:34:51.940399 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:51.940369 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:52.249180 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:52.249140 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" event={"ID":"0719fc1d-2dea-450e-b161-8e2361629cf0","Type":"ContainerStarted","Data":"aaf2ed8156ec5619f76f402531d109dfac3aa437bf50c7847db52d63feac42b8"} Sep 06 10:34:52.249387 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:52.249368 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:52.250734 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:52.250714 2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-m7ftd container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 06 10:34:52.250793 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:52.250764 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" podUID="0719fc1d-2dea-450e-b161-8e2361629cf0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 06 10:34:52.300992 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:52.300940 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" podStartSLOduration=2.03555827 podStartE2EDuration="4.300925511s" totalImagesPullingTime="2.265367241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:34:52.300817705 +0000 UTC m=+285.883307338" watchObservedRunningTime="2026-09-06 10:34:52.300925511 +0000 UTC m=+285.883415142" Sep 06 10:34:53.089730 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:53.089699 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:53.255229 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:53.255202 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-m7ftd" Sep 06 10:34:53.260139 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:53.260096 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:53.348221 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:53.348143 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:53.412084 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:53.412054 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:34:57.211111 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:57.211082 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-zn5wj" Sep 06 10:34:59.231844 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:34:59.231806 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-tvq92" Sep 06 10:35:06.857998 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:06.857971 2574 kubelet.go:1648] "Image garbage collection succeeded" Sep 06 10:35:15.822173 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:15.822141 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:35:15.880487 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:15.880458 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:35:42.038056 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:42.038012 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:35:42.130604 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:42.130566 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:35:45.305331 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.305290 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8"] Sep 06 10:35:45.308971 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.308949 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:45.311202 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.311172 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 06 10:35:45.311202 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.311190 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 06 10:35:45.311926 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.311911 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jqnm5\"" Sep 06 10:35:45.316578 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.316558 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8"] Sep 06 10:35:45.408502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.408468 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg"] Sep 06 10:35:45.411919 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.411902 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:45.414251 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.414220 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f764f8d-c42a-4a65-938a-249b702066e7-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8\" (UID: \"2f764f8d-c42a-4a65-938a-249b702066e7\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:45.414375 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.414260 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw55m\" (UniqueName: \"kubernetes.io/projected/2f764f8d-c42a-4a65-938a-249b702066e7-kube-api-access-kw55m\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8\" (UID: \"2f764f8d-c42a-4a65-938a-249b702066e7\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:45.414375 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.414350 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f764f8d-c42a-4a65-938a-249b702066e7-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8\" (UID: \"2f764f8d-c42a-4a65-938a-249b702066e7\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:45.420917 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.420891 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg"] Sep 06 10:35:45.509465 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.509419 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64"] Sep 06 10:35:45.514410 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.514382 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:45.515034 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.515006 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c57ab36c-d0f5-432b-93da-aec1ce34eef3-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg\" (UID: \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:45.515612 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.515241 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67hjr\" (UniqueName: \"kubernetes.io/projected/c57ab36c-d0f5-432b-93da-aec1ce34eef3-kube-api-access-67hjr\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg\" (UID: \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:45.515612 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.515341 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f764f8d-c42a-4a65-938a-249b702066e7-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8\" (UID: \"2f764f8d-c42a-4a65-938a-249b702066e7\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:45.515612 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.515374 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c57ab36c-d0f5-432b-93da-aec1ce34eef3-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg\" (UID: \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:45.515612 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.515416 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw55m\" (UniqueName: \"kubernetes.io/projected/2f764f8d-c42a-4a65-938a-249b702066e7-kube-api-access-kw55m\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8\" (UID: \"2f764f8d-c42a-4a65-938a-249b702066e7\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:45.515612 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.515463 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f764f8d-c42a-4a65-938a-249b702066e7-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8\" (UID: \"2f764f8d-c42a-4a65-938a-249b702066e7\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:45.515993 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.515874 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f764f8d-c42a-4a65-938a-249b702066e7-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8\" (UID: \"2f764f8d-c42a-4a65-938a-249b702066e7\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:45.516226 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.516202 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f764f8d-c42a-4a65-938a-249b702066e7-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8\" (UID: \"2f764f8d-c42a-4a65-938a-249b702066e7\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:45.518015 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.517992 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64"] Sep 06 10:35:45.523239 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.523221 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw55m\" (UniqueName: \"kubernetes.io/projected/2f764f8d-c42a-4a65-938a-249b702066e7-kube-api-access-kw55m\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8\" (UID: \"2f764f8d-c42a-4a65-938a-249b702066e7\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:45.607805 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.607728 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8"] Sep 06 10:35:45.611410 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.611386 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:45.616448 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.616414 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c57ab36c-d0f5-432b-93da-aec1ce34eef3-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg\" (UID: \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:45.616579 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.616466 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67hjr\" (UniqueName: \"kubernetes.io/projected/c57ab36c-d0f5-432b-93da-aec1ce34eef3-kube-api-access-67hjr\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg\" (UID: \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:45.616579 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.616541 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gq7r\" (UniqueName: \"kubernetes.io/projected/2ff6b04a-a675-4532-9c46-bca1e776a363-kube-api-access-2gq7r\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64\" (UID: \"2ff6b04a-a675-4532-9c46-bca1e776a363\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:45.616579 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.616571 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c57ab36c-d0f5-432b-93da-aec1ce34eef3-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg\" (UID: \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:45.616745 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.616613 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ff6b04a-a675-4532-9c46-bca1e776a363-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64\" (UID: \"2ff6b04a-a675-4532-9c46-bca1e776a363\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:45.616745 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.616637 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ff6b04a-a675-4532-9c46-bca1e776a363-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64\" (UID: \"2ff6b04a-a675-4532-9c46-bca1e776a363\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:45.616863 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.616830 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c57ab36c-d0f5-432b-93da-aec1ce34eef3-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg\" (UID: \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:45.617025 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.617000 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c57ab36c-d0f5-432b-93da-aec1ce34eef3-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg\" (UID: \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:45.618816 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.618798 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:45.620089 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.620067 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8"] Sep 06 10:35:45.624479 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.624454 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67hjr\" (UniqueName: \"kubernetes.io/projected/c57ab36c-d0f5-432b-93da-aec1ce34eef3-kube-api-access-67hjr\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg\" (UID: \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:45.717867 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.717834 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gq7r\" (UniqueName: \"kubernetes.io/projected/2ff6b04a-a675-4532-9c46-bca1e776a363-kube-api-access-2gq7r\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64\" (UID: \"2ff6b04a-a675-4532-9c46-bca1e776a363\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:45.717867 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.717875 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8\" (UID: \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:45.718111 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.717906 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ff6b04a-a675-4532-9c46-bca1e776a363-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64\" (UID: \"2ff6b04a-a675-4532-9c46-bca1e776a363\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:45.718111 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.717959 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ff6b04a-a675-4532-9c46-bca1e776a363-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64\" (UID: \"2ff6b04a-a675-4532-9c46-bca1e776a363\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:45.718111 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.718045 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bltkr\" (UniqueName: \"kubernetes.io/projected/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-kube-api-access-bltkr\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8\" (UID: \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:45.718111 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.718092 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8\" (UID: \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:45.718337 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.718315 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ff6b04a-a675-4532-9c46-bca1e776a363-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64\" (UID: \"2ff6b04a-a675-4532-9c46-bca1e776a363\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:45.718375 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.718342 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ff6b04a-a675-4532-9c46-bca1e776a363-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64\" (UID: \"2ff6b04a-a675-4532-9c46-bca1e776a363\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:45.721724 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.721706 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:45.726484 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.726464 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gq7r\" (UniqueName: \"kubernetes.io/projected/2ff6b04a-a675-4532-9c46-bca1e776a363-kube-api-access-2gq7r\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64\" (UID: \"2ff6b04a-a675-4532-9c46-bca1e776a363\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:45.818894 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.818857 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bltkr\" (UniqueName: \"kubernetes.io/projected/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-kube-api-access-bltkr\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8\" (UID: \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:45.819047 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.818908 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8\" (UID: \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:45.819047 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.818990 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8\" (UID: \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:45.819349 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.819329 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8\" (UID: \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:45.819396 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.819362 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8\" (UID: \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:45.828282 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.828260 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bltkr\" (UniqueName: \"kubernetes.io/projected/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-kube-api-access-bltkr\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8\" (UID: \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:45.837022 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.836999 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:45.895615 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:35:45.895583 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f764f8d_c42a_4a65_938a_249b702066e7.slice/crio-ef95005ad04bd8c19a01d7012863997a6d6c4bd1249b953cb62420bc0d334a1c WatchSource:0}: Error finding container ef95005ad04bd8c19a01d7012863997a6d6c4bd1249b953cb62420bc0d334a1c: Status 404 returned error can't find the container with id ef95005ad04bd8c19a01d7012863997a6d6c4bd1249b953cb62420bc0d334a1c Sep 06 10:35:45.896673 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.896645 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8"] Sep 06 10:35:45.920849 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:45.920828 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:46.000561 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.000484 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg"] Sep 06 10:35:46.002693 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:35:46.002665 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc57ab36c_d0f5_432b_93da_aec1ce34eef3.slice/crio-3a56a7dbd416f14ad4c64fa8e583a0dc96acae8c3b2ffb588ff3fbe5c158df39 WatchSource:0}: Error finding container 3a56a7dbd416f14ad4c64fa8e583a0dc96acae8c3b2ffb588ff3fbe5c158df39: Status 404 returned error can't find the container with id 3a56a7dbd416f14ad4c64fa8e583a0dc96acae8c3b2ffb588ff3fbe5c158df39 Sep 06 10:35:46.109440 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.109411 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64"] Sep 06 10:35:46.111344 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:35:46.111300 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff6b04a_a675_4532_9c46_bca1e776a363.slice/crio-4e6c7d394e74ed107e0d897ddb8ac5a38806be130db40a74346f64ede1a8b958 WatchSource:0}: Error finding container 4e6c7d394e74ed107e0d897ddb8ac5a38806be130db40a74346f64ede1a8b958: Status 404 returned error can't find the container with id 4e6c7d394e74ed107e0d897ddb8ac5a38806be130db40a74346f64ede1a8b958 Sep 06 10:35:46.197727 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.197703 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8"] Sep 06 10:35:46.200262 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:35:46.200211 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2a1f1b_1f37_4a61_9aa3_7474634636bf.slice/crio-d5049748bda288bb1759e8ff5cb2b1ab64dda222f702a1adf367cd4346af376c WatchSource:0}: Error finding container d5049748bda288bb1759e8ff5cb2b1ab64dda222f702a1adf367cd4346af376c: Status 404 returned error can't find the container with id d5049748bda288bb1759e8ff5cb2b1ab64dda222f702a1adf367cd4346af376c Sep 06 10:35:46.523436 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.523344 2574 generic.go:356] "Generic (PLEG): container finished" podID="cd2a1f1b-1f37-4a61-9aa3-7474634636bf" containerID="2b8b5a84db03bda077098d66dcd36886718e88f63b626eaea681732bf9aecdf1" exitCode=0 Sep 06 10:35:46.523837 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.523432 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" event={"ID":"cd2a1f1b-1f37-4a61-9aa3-7474634636bf","Type":"ContainerDied","Data":"2b8b5a84db03bda077098d66dcd36886718e88f63b626eaea681732bf9aecdf1"} Sep 06 10:35:46.523837 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.523492 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" event={"ID":"cd2a1f1b-1f37-4a61-9aa3-7474634636bf","Type":"ContainerStarted","Data":"d5049748bda288bb1759e8ff5cb2b1ab64dda222f702a1adf367cd4346af376c"} Sep 06 10:35:46.524590 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.524569 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 10:35:46.525197 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.525101 2574 generic.go:356] "Generic (PLEG): container finished" podID="2ff6b04a-a675-4532-9c46-bca1e776a363" containerID="e7a52ab7ee4061fc16419d662401874008e41c16c33106883cc7350b1479ca4e" exitCode=0 Sep 06 10:35:46.525197 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.525161 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" event={"ID":"2ff6b04a-a675-4532-9c46-bca1e776a363","Type":"ContainerDied","Data":"e7a52ab7ee4061fc16419d662401874008e41c16c33106883cc7350b1479ca4e"} Sep 06 10:35:46.525320 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.525206 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" event={"ID":"2ff6b04a-a675-4532-9c46-bca1e776a363","Type":"ContainerStarted","Data":"4e6c7d394e74ed107e0d897ddb8ac5a38806be130db40a74346f64ede1a8b958"} Sep 06 10:35:46.527171 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.527117 2574 generic.go:356] "Generic (PLEG): container finished" podID="c57ab36c-d0f5-432b-93da-aec1ce34eef3" containerID="5d2720e407e327cbd6f91efbf6bebe1c95ab7cde69d51a50e3ab32988285b843" exitCode=0 Sep 06 10:35:46.527274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.527214 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" event={"ID":"c57ab36c-d0f5-432b-93da-aec1ce34eef3","Type":"ContainerDied","Data":"5d2720e407e327cbd6f91efbf6bebe1c95ab7cde69d51a50e3ab32988285b843"} Sep 06 10:35:46.527274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.527253 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" event={"ID":"c57ab36c-d0f5-432b-93da-aec1ce34eef3","Type":"ContainerStarted","Data":"3a56a7dbd416f14ad4c64fa8e583a0dc96acae8c3b2ffb588ff3fbe5c158df39"} Sep 06 10:35:46.529028 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.529004 2574 generic.go:356] "Generic (PLEG): container finished" podID="2f764f8d-c42a-4a65-938a-249b702066e7" containerID="4ec758018634e4b98069f7ba4879d958e91eb6269bd8deffbbf364b3b4fb0206" exitCode=0 Sep 06 10:35:46.529160 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.529049 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" event={"ID":"2f764f8d-c42a-4a65-938a-249b702066e7","Type":"ContainerDied","Data":"4ec758018634e4b98069f7ba4879d958e91eb6269bd8deffbbf364b3b4fb0206"} Sep 06 10:35:46.529160 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:46.529081 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" event={"ID":"2f764f8d-c42a-4a65-938a-249b702066e7","Type":"ContainerStarted","Data":"ef95005ad04bd8c19a01d7012863997a6d6c4bd1249b953cb62420bc0d334a1c"} Sep 06 10:35:47.538370 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:47.538329 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" event={"ID":"2f764f8d-c42a-4a65-938a-249b702066e7","Type":"ContainerStarted","Data":"d1f1e714177fd5c22354c6df62bc1c366185ad627abdf05d4da49eb8bc5e389b"} Sep 06 10:35:47.540496 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:47.540470 2574 generic.go:356] "Generic (PLEG): container finished" podID="cd2a1f1b-1f37-4a61-9aa3-7474634636bf" containerID="32ae37fdbeaa59a88ad2aa1b12fdded82a44eb1ebf88b2f65afcdee6e8463582" exitCode=0 Sep 06 10:35:47.540627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:47.540553 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" event={"ID":"cd2a1f1b-1f37-4a61-9aa3-7474634636bf","Type":"ContainerDied","Data":"32ae37fdbeaa59a88ad2aa1b12fdded82a44eb1ebf88b2f65afcdee6e8463582"} Sep 06 10:35:48.551593 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:48.551558 2574 generic.go:356] "Generic (PLEG): container finished" podID="2ff6b04a-a675-4532-9c46-bca1e776a363" containerID="e7281733bac842e2b79924ddcff5fffcab89fd9a3731c9840de5ced759c338aa" exitCode=0 Sep 06 10:35:48.551989 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:48.551644 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" event={"ID":"2ff6b04a-a675-4532-9c46-bca1e776a363","Type":"ContainerDied","Data":"e7281733bac842e2b79924ddcff5fffcab89fd9a3731c9840de5ced759c338aa"} Sep 06 10:35:48.553698 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:48.553676 2574 generic.go:356] "Generic (PLEG): container finished" podID="c57ab36c-d0f5-432b-93da-aec1ce34eef3" containerID="47d70614bbfdfd95ca23a3f377ba4263bb0570991c75c69b92ce7838e758ae0e" exitCode=0 Sep 06 10:35:48.553788 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:48.553760 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" event={"ID":"c57ab36c-d0f5-432b-93da-aec1ce34eef3","Type":"ContainerDied","Data":"47d70614bbfdfd95ca23a3f377ba4263bb0570991c75c69b92ce7838e758ae0e"} Sep 06 10:35:48.555858 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:48.555826 2574 generic.go:356] "Generic (PLEG): container finished" podID="2f764f8d-c42a-4a65-938a-249b702066e7" containerID="d1f1e714177fd5c22354c6df62bc1c366185ad627abdf05d4da49eb8bc5e389b" exitCode=0 Sep 06 10:35:48.555937 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:48.555901 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" event={"ID":"2f764f8d-c42a-4a65-938a-249b702066e7","Type":"ContainerDied","Data":"d1f1e714177fd5c22354c6df62bc1c366185ad627abdf05d4da49eb8bc5e389b"} Sep 06 10:35:48.558428 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:48.558404 2574 generic.go:356] "Generic (PLEG): container finished" podID="cd2a1f1b-1f37-4a61-9aa3-7474634636bf" containerID="6a66f86ef715ca3ebf891c75b85eb6e21db7c70d14f63ee02977866aac96e550" exitCode=0 Sep 06 10:35:48.558504 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:48.558437 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" event={"ID":"cd2a1f1b-1f37-4a61-9aa3-7474634636bf","Type":"ContainerDied","Data":"6a66f86ef715ca3ebf891c75b85eb6e21db7c70d14f63ee02977866aac96e550"} Sep 06 10:35:49.565450 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.565420 2574 generic.go:356] "Generic (PLEG): container finished" podID="c57ab36c-d0f5-432b-93da-aec1ce34eef3" containerID="7b014fb5c61f76a8b5ba92c944663f80654d754aba3ab0108865270116c64e8e" exitCode=0 Sep 06 10:35:49.565837 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.565505 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" event={"ID":"c57ab36c-d0f5-432b-93da-aec1ce34eef3","Type":"ContainerDied","Data":"7b014fb5c61f76a8b5ba92c944663f80654d754aba3ab0108865270116c64e8e"} Sep 06 10:35:49.567620 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.567600 2574 generic.go:356] "Generic (PLEG): container finished" podID="2f764f8d-c42a-4a65-938a-249b702066e7" containerID="91c1ac9d658741682a6616d64c18a4637f6bee840a124ad330c97bd56c3eb877" exitCode=0 Sep 06 10:35:49.567732 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.567680 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" event={"ID":"2f764f8d-c42a-4a65-938a-249b702066e7","Type":"ContainerDied","Data":"91c1ac9d658741682a6616d64c18a4637f6bee840a124ad330c97bd56c3eb877"} Sep 06 10:35:49.569788 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.569766 2574 generic.go:356] "Generic (PLEG): container finished" podID="2ff6b04a-a675-4532-9c46-bca1e776a363" containerID="1bbd30218d19ca4af2e144b38a0e2ff90ef5f2392587355e526383e7e098d8a9" exitCode=0 Sep 06 10:35:49.569919 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.569896 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" event={"ID":"2ff6b04a-a675-4532-9c46-bca1e776a363","Type":"ContainerDied","Data":"1bbd30218d19ca4af2e144b38a0e2ff90ef5f2392587355e526383e7e098d8a9"} Sep 06 10:35:49.700786 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.700761 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:49.751550 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.751519 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-bundle\") pod \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\" (UID: \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\") " Sep 06 10:35:49.751700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.751615 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bltkr\" (UniqueName: \"kubernetes.io/projected/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-kube-api-access-bltkr\") pod \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\" (UID: \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\") " Sep 06 10:35:49.751700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.751656 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-util\") pod \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\" (UID: \"cd2a1f1b-1f37-4a61-9aa3-7474634636bf\") " Sep 06 10:35:49.752059 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.752031 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-bundle" (OuterVolumeSpecName: "bundle") pod "cd2a1f1b-1f37-4a61-9aa3-7474634636bf" (UID: "cd2a1f1b-1f37-4a61-9aa3-7474634636bf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:35:49.753710 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.753686 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-kube-api-access-bltkr" (OuterVolumeSpecName: "kube-api-access-bltkr") pod "cd2a1f1b-1f37-4a61-9aa3-7474634636bf" (UID: "cd2a1f1b-1f37-4a61-9aa3-7474634636bf"). InnerVolumeSpecName "kube-api-access-bltkr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:35:49.756946 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.756906 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-util" (OuterVolumeSpecName: "util") pod "cd2a1f1b-1f37-4a61-9aa3-7474634636bf" (UID: "cd2a1f1b-1f37-4a61-9aa3-7474634636bf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:35:49.853321 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.853223 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bltkr\" (UniqueName: \"kubernetes.io/projected/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-kube-api-access-bltkr\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:35:49.853321 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.853264 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-util\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:35:49.853321 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:49.853275 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd2a1f1b-1f37-4a61-9aa3-7474634636bf-bundle\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:35:50.577927 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.577886 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" event={"ID":"cd2a1f1b-1f37-4a61-9aa3-7474634636bf","Type":"ContainerDied","Data":"d5049748bda288bb1759e8ff5cb2b1ab64dda222f702a1adf367cd4346af376c"} Sep 06 10:35:50.577927 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.577912 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cf6xr8" Sep 06 10:35:50.578374 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.577944 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5049748bda288bb1759e8ff5cb2b1ab64dda222f702a1adf367cd4346af376c" Sep 06 10:35:50.725550 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.725525 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:50.760227 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.760198 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c57ab36c-d0f5-432b-93da-aec1ce34eef3-util\") pod \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\" (UID: \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\") " Sep 06 10:35:50.760359 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.760304 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c57ab36c-d0f5-432b-93da-aec1ce34eef3-bundle\") pod \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\" (UID: \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\") " Sep 06 10:35:50.760435 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.760373 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67hjr\" (UniqueName: \"kubernetes.io/projected/c57ab36c-d0f5-432b-93da-aec1ce34eef3-kube-api-access-67hjr\") pod \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\" (UID: \"c57ab36c-d0f5-432b-93da-aec1ce34eef3\") " Sep 06 10:35:50.760862 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.760835 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c57ab36c-d0f5-432b-93da-aec1ce34eef3-bundle" (OuterVolumeSpecName: "bundle") pod "c57ab36c-d0f5-432b-93da-aec1ce34eef3" (UID: "c57ab36c-d0f5-432b-93da-aec1ce34eef3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:35:50.762437 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.762408 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c57ab36c-d0f5-432b-93da-aec1ce34eef3-kube-api-access-67hjr" (OuterVolumeSpecName: "kube-api-access-67hjr") pod "c57ab36c-d0f5-432b-93da-aec1ce34eef3" (UID: "c57ab36c-d0f5-432b-93da-aec1ce34eef3"). InnerVolumeSpecName "kube-api-access-67hjr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:35:50.765897 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.765873 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c57ab36c-d0f5-432b-93da-aec1ce34eef3-util" (OuterVolumeSpecName: "util") pod "c57ab36c-d0f5-432b-93da-aec1ce34eef3" (UID: "c57ab36c-d0f5-432b-93da-aec1ce34eef3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:35:50.773704 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.773683 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:50.777244 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.777224 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:50.861874 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.861783 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f764f8d-c42a-4a65-938a-249b702066e7-util\") pod \"2f764f8d-c42a-4a65-938a-249b702066e7\" (UID: \"2f764f8d-c42a-4a65-938a-249b702066e7\") " Sep 06 10:35:50.861874 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.861868 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw55m\" (UniqueName: \"kubernetes.io/projected/2f764f8d-c42a-4a65-938a-249b702066e7-kube-api-access-kw55m\") pod \"2f764f8d-c42a-4a65-938a-249b702066e7\" (UID: \"2f764f8d-c42a-4a65-938a-249b702066e7\") " Sep 06 10:35:50.862067 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.861916 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ff6b04a-a675-4532-9c46-bca1e776a363-bundle\") pod \"2ff6b04a-a675-4532-9c46-bca1e776a363\" (UID: \"2ff6b04a-a675-4532-9c46-bca1e776a363\") " Sep 06 10:35:50.862067 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.861930 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ff6b04a-a675-4532-9c46-bca1e776a363-util\") pod \"2ff6b04a-a675-4532-9c46-bca1e776a363\" (UID: \"2ff6b04a-a675-4532-9c46-bca1e776a363\") " Sep 06 10:35:50.862067 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.861950 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gq7r\" (UniqueName: \"kubernetes.io/projected/2ff6b04a-a675-4532-9c46-bca1e776a363-kube-api-access-2gq7r\") pod \"2ff6b04a-a675-4532-9c46-bca1e776a363\" (UID: \"2ff6b04a-a675-4532-9c46-bca1e776a363\") " Sep 06 10:35:50.862067 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.861986 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f764f8d-c42a-4a65-938a-249b702066e7-bundle\") pod \"2f764f8d-c42a-4a65-938a-249b702066e7\" (UID: \"2f764f8d-c42a-4a65-938a-249b702066e7\") " Sep 06 10:35:50.862304 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.862254 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c57ab36c-d0f5-432b-93da-aec1ce34eef3-bundle\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:35:50.862304 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.862276 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67hjr\" (UniqueName: \"kubernetes.io/projected/c57ab36c-d0f5-432b-93da-aec1ce34eef3-kube-api-access-67hjr\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:35:50.862304 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.862289 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c57ab36c-d0f5-432b-93da-aec1ce34eef3-util\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:35:50.862634 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.862577 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f764f8d-c42a-4a65-938a-249b702066e7-bundle" (OuterVolumeSpecName: "bundle") pod "2f764f8d-c42a-4a65-938a-249b702066e7" (UID: "2f764f8d-c42a-4a65-938a-249b702066e7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:35:50.862813 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.862783 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff6b04a-a675-4532-9c46-bca1e776a363-bundle" (OuterVolumeSpecName: "bundle") pod "2ff6b04a-a675-4532-9c46-bca1e776a363" (UID: "2ff6b04a-a675-4532-9c46-bca1e776a363"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:35:50.864107 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.864079 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ff6b04a-a675-4532-9c46-bca1e776a363-kube-api-access-2gq7r" (OuterVolumeSpecName: "kube-api-access-2gq7r") pod "2ff6b04a-a675-4532-9c46-bca1e776a363" (UID: "2ff6b04a-a675-4532-9c46-bca1e776a363"). InnerVolumeSpecName "kube-api-access-2gq7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:35:50.864232 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.864153 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f764f8d-c42a-4a65-938a-249b702066e7-kube-api-access-kw55m" (OuterVolumeSpecName: "kube-api-access-kw55m") pod "2f764f8d-c42a-4a65-938a-249b702066e7" (UID: "2f764f8d-c42a-4a65-938a-249b702066e7"). InnerVolumeSpecName "kube-api-access-kw55m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:35:50.870075 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.870030 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff6b04a-a675-4532-9c46-bca1e776a363-util" (OuterVolumeSpecName: "util") pod "2ff6b04a-a675-4532-9c46-bca1e776a363" (UID: "2ff6b04a-a675-4532-9c46-bca1e776a363"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:35:50.870836 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.870817 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f764f8d-c42a-4a65-938a-249b702066e7-util" (OuterVolumeSpecName: "util") pod "2f764f8d-c42a-4a65-938a-249b702066e7" (UID: "2f764f8d-c42a-4a65-938a-249b702066e7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:35:50.963095 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.963065 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ff6b04a-a675-4532-9c46-bca1e776a363-bundle\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:35:50.963095 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.963092 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ff6b04a-a675-4532-9c46-bca1e776a363-util\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:35:50.963295 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.963102 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gq7r\" (UniqueName: \"kubernetes.io/projected/2ff6b04a-a675-4532-9c46-bca1e776a363-kube-api-access-2gq7r\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:35:50.963295 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.963112 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f764f8d-c42a-4a65-938a-249b702066e7-bundle\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:35:50.963295 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.963120 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f764f8d-c42a-4a65-938a-249b702066e7-util\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:35:50.963295 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:50.963139 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw55m\" (UniqueName: \"kubernetes.io/projected/2f764f8d-c42a-4a65-938a-249b702066e7-kube-api-access-kw55m\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:35:51.585038 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:51.585003 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" event={"ID":"c57ab36c-d0f5-432b-93da-aec1ce34eef3","Type":"ContainerDied","Data":"3a56a7dbd416f14ad4c64fa8e583a0dc96acae8c3b2ffb588ff3fbe5c158df39"} Sep 06 10:35:51.585038 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:51.585023 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbks6hg" Sep 06 10:35:51.585561 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:51.585057 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a56a7dbd416f14ad4c64fa8e583a0dc96acae8c3b2ffb588ff3fbe5c158df39" Sep 06 10:35:51.587397 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:51.587304 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" Sep 06 10:35:51.587397 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:51.587300 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0krmq8" event={"ID":"2f764f8d-c42a-4a65-938a-249b702066e7","Type":"ContainerDied","Data":"ef95005ad04bd8c19a01d7012863997a6d6c4bd1249b953cb62420bc0d334a1c"} Sep 06 10:35:51.587608 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:51.587435 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef95005ad04bd8c19a01d7012863997a6d6c4bd1249b953cb62420bc0d334a1c" Sep 06 10:35:51.589611 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:51.589590 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" event={"ID":"2ff6b04a-a675-4532-9c46-bca1e776a363","Type":"ContainerDied","Data":"4e6c7d394e74ed107e0d897ddb8ac5a38806be130db40a74346f64ede1a8b958"} Sep 06 10:35:51.589611 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:51.589611 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d8705368g64" Sep 06 10:35:51.589611 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:51.589621 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e6c7d394e74ed107e0d897ddb8ac5a38806be130db40a74346f64ede1a8b958" Sep 06 10:35:55.218808 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:55.218767 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:35:58.021732 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:58.021699 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:35:59.626000 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:35:59.625919 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:00.008693 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:00.008628 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:00.277803 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:00.277685 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:01.649221 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:01.649162 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:02.624262 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:02.624225 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:02.732561 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:02.732528 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:03.033673 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:03.033640 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:03.153388 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:03.153354 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:03.426738 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:03.426703 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:03.535353 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:03.535311 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:03.985189 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:03.985118 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:07.828074 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:07.828031 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:07.924795 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:07.924762 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:11.219201 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:11.219171 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:11.319397 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:11.319366 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:13.826017 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:13.825971 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:14.100354 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:14.100272 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:15.668727 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:15.668694 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:15.773325 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:15.773291 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:16.020257 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:16.020147 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:18.525795 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:18.525761 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:20.466432 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:20.466399 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:20.561971 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:20.561931 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:20.674409 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:20.674343 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:27.113374 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:27.113339 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:27.213338 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:27.213305 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:30.317734 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:30.317694 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:30.415315 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:30.415282 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:34.732640 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:34.732607 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:34.833054 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:34.833019 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:34.933890 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:34.933858 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:35.034532 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:35.034454 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:39.859688 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:39.859646 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:39.960445 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:39.960411 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:40.059709 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:40.059674 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:53.448344 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:53.448277 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:53.593104 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:53.593068 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:55.919842 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.919807 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5556bd56b8-hfvh8"] Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920146 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c57ab36c-d0f5-432b-93da-aec1ce34eef3" containerName="extract" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920165 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c57ab36c-d0f5-432b-93da-aec1ce34eef3" containerName="extract" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920180 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f764f8d-c42a-4a65-938a-249b702066e7" containerName="util" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920186 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f764f8d-c42a-4a65-938a-249b702066e7" containerName="util" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920192 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd2a1f1b-1f37-4a61-9aa3-7474634636bf" containerName="util" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920198 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd2a1f1b-1f37-4a61-9aa3-7474634636bf" containerName="util" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920206 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd2a1f1b-1f37-4a61-9aa3-7474634636bf" containerName="pull" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920211 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd2a1f1b-1f37-4a61-9aa3-7474634636bf" containerName="pull" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920217 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ff6b04a-a675-4532-9c46-bca1e776a363" containerName="pull" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920222 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ff6b04a-a675-4532-9c46-bca1e776a363" containerName="pull" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920230 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c57ab36c-d0f5-432b-93da-aec1ce34eef3" containerName="util" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920235 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c57ab36c-d0f5-432b-93da-aec1ce34eef3" containerName="util" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920240 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd2a1f1b-1f37-4a61-9aa3-7474634636bf" containerName="extract" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920245 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd2a1f1b-1f37-4a61-9aa3-7474634636bf" containerName="extract" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920251 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c57ab36c-d0f5-432b-93da-aec1ce34eef3" containerName="pull" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920255 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c57ab36c-d0f5-432b-93da-aec1ce34eef3" containerName="pull" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920263 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ff6b04a-a675-4532-9c46-bca1e776a363" containerName="extract" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920269 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ff6b04a-a675-4532-9c46-bca1e776a363" containerName="extract" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920276 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f764f8d-c42a-4a65-938a-249b702066e7" containerName="extract" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920281 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f764f8d-c42a-4a65-938a-249b702066e7" containerName="extract" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920290 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f764f8d-c42a-4a65-938a-249b702066e7" containerName="pull" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920296 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f764f8d-c42a-4a65-938a-249b702066e7" containerName="pull" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920303 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ff6b04a-a675-4532-9c46-bca1e776a363" containerName="util" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920308 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ff6b04a-a675-4532-9c46-bca1e776a363" containerName="util" Sep 06 10:36:55.920345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920362 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f764f8d-c42a-4a65-938a-249b702066e7" containerName="extract" Sep 06 10:36:55.921429 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920371 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd2a1f1b-1f37-4a61-9aa3-7474634636bf" containerName="extract" Sep 06 10:36:55.921429 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920379 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ff6b04a-a675-4532-9c46-bca1e776a363" containerName="extract" Sep 06 10:36:55.921429 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.920385 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c57ab36c-d0f5-432b-93da-aec1ce34eef3" containerName="extract" Sep 06 10:36:55.923605 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.923584 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:55.932814 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.932789 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5556bd56b8-hfvh8"] Sep 06 10:36:55.977671 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:55.977642 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:56.015283 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.015254 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8413a709-bee7-4265-ac32-9f867297c28e-console-serving-cert\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.015436 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.015285 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmmqj\" (UniqueName: \"kubernetes.io/projected/8413a709-bee7-4265-ac32-9f867297c28e-kube-api-access-nmmqj\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.015436 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.015306 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8413a709-bee7-4265-ac32-9f867297c28e-console-oauth-config\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.015436 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.015363 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8413a709-bee7-4265-ac32-9f867297c28e-service-ca\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.015436 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.015383 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8413a709-bee7-4265-ac32-9f867297c28e-oauth-serving-cert\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.015436 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.015399 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8413a709-bee7-4265-ac32-9f867297c28e-console-config\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.015647 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.015518 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8413a709-bee7-4265-ac32-9f867297c28e-trusted-ca-bundle\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.032142 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.032100 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:36:56.116720 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.116682 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8413a709-bee7-4265-ac32-9f867297c28e-trusted-ca-bundle\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.116851 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.116740 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8413a709-bee7-4265-ac32-9f867297c28e-console-serving-cert\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.116894 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.116843 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmmqj\" (UniqueName: \"kubernetes.io/projected/8413a709-bee7-4265-ac32-9f867297c28e-kube-api-access-nmmqj\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.116894 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.116875 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8413a709-bee7-4265-ac32-9f867297c28e-console-oauth-config\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.116976 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.116902 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8413a709-bee7-4265-ac32-9f867297c28e-service-ca\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.116976 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.116928 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8413a709-bee7-4265-ac32-9f867297c28e-oauth-serving-cert\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.116976 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.116953 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8413a709-bee7-4265-ac32-9f867297c28e-console-config\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.117694 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.117669 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8413a709-bee7-4265-ac32-9f867297c28e-console-config\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.117694 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.117690 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8413a709-bee7-4265-ac32-9f867297c28e-trusted-ca-bundle\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.117839 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.117718 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8413a709-bee7-4265-ac32-9f867297c28e-service-ca\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.117839 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.117687 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8413a709-bee7-4265-ac32-9f867297c28e-oauth-serving-cert\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.119341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.119312 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8413a709-bee7-4265-ac32-9f867297c28e-console-oauth-config\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.119436 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.119359 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8413a709-bee7-4265-ac32-9f867297c28e-console-serving-cert\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.124974 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.124952 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmmqj\" (UniqueName: \"kubernetes.io/projected/8413a709-bee7-4265-ac32-9f867297c28e-kube-api-access-nmmqj\") pod \"console-5556bd56b8-hfvh8\" (UID: \"8413a709-bee7-4265-ac32-9f867297c28e\") " pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.233286 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.233191 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:36:56.505654 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.505629 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5556bd56b8-hfvh8"] Sep 06 10:36:56.507042 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:36:56.507017 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8413a709_bee7_4265_ac32_9f867297c28e.slice/crio-b18e5beda5fd6a79aa2d5652df21b90b1ba9f26cae7a62e7ad6f639319231189 WatchSource:0}: Error finding container b18e5beda5fd6a79aa2d5652df21b90b1ba9f26cae7a62e7ad6f639319231189: Status 404 returned error can't find the container with id b18e5beda5fd6a79aa2d5652df21b90b1ba9f26cae7a62e7ad6f639319231189 Sep 06 10:36:56.928458 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.928421 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5556bd56b8-hfvh8" event={"ID":"8413a709-bee7-4265-ac32-9f867297c28e","Type":"ContainerStarted","Data":"14c1680c33704c74a9cc420f428410ec4de43dae17b4bac3e0e72c3161873687"} Sep 06 10:36:56.928907 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.928473 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5556bd56b8-hfvh8" event={"ID":"8413a709-bee7-4265-ac32-9f867297c28e","Type":"ContainerStarted","Data":"b18e5beda5fd6a79aa2d5652df21b90b1ba9f26cae7a62e7ad6f639319231189"} Sep 06 10:36:56.947052 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:36:56.946994 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5556bd56b8-hfvh8" podStartSLOduration=1.946978601 podStartE2EDuration="1.946978601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:36:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:36:56.944885861 +0000 UTC m=+410.527375506" watchObservedRunningTime="2026-09-06 10:36:56.946978601 +0000 UTC m=+410.529468231" Sep 06 10:37:06.233576 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:06.233537 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:37:06.234051 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:06.233606 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:37:06.238248 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:06.238223 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:37:06.935000 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:06.934970 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:06.984663 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:06.984625 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5556bd56b8-hfvh8" Sep 06 10:37:07.038875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:07.038842 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7656c95999-6pvms"] Sep 06 10:37:07.047568 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:07.047537 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:07.112339 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:07.112304 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:07.212621 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:07.212543 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:07.313015 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:07.312978 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:07.753464 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:07.753434 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:07.831108 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:07.831073 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:21.585995 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:21.585937 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:24.474312 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:24.474276 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:24.571293 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:24.571261 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:24.941064 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:24.941038 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:32.064545 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.064442 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7656c95999-6pvms" podUID="d0c3c91e-aeb1-4843-a8ab-0882781d767f" containerName="console" containerID="cri-o://af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae" gracePeriod=15 Sep 06 10:37:32.309029 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.309007 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:37:32.421829 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.421800 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-config\") pod \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " Sep 06 10:37:32.422012 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.421848 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-oauth-serving-cert\") pod \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " Sep 06 10:37:32.422012 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.421885 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-serving-cert\") pod \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " Sep 06 10:37:32.422012 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.421903 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-trusted-ca-bundle\") pod \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " Sep 06 10:37:32.422012 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.422002 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf69m\" (UniqueName: \"kubernetes.io/projected/d0c3c91e-aeb1-4843-a8ab-0882781d767f-kube-api-access-xf69m\") pod \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " Sep 06 10:37:32.422276 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.422040 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-oauth-config\") pod \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " Sep 06 10:37:32.422276 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.422075 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-service-ca\") pod \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\" (UID: \"d0c3c91e-aeb1-4843-a8ab-0882781d767f\") " Sep 06 10:37:32.422424 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.422398 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-config" (OuterVolumeSpecName: "console-config") pod "d0c3c91e-aeb1-4843-a8ab-0882781d767f" (UID: "d0c3c91e-aeb1-4843-a8ab-0882781d767f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:37:32.422481 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.422422 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d0c3c91e-aeb1-4843-a8ab-0882781d767f" (UID: "d0c3c91e-aeb1-4843-a8ab-0882781d767f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:37:32.422544 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.422496 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d0c3c91e-aeb1-4843-a8ab-0882781d767f" (UID: "d0c3c91e-aeb1-4843-a8ab-0882781d767f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:37:32.422544 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.422526 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-service-ca" (OuterVolumeSpecName: "service-ca") pod "d0c3c91e-aeb1-4843-a8ab-0882781d767f" (UID: "d0c3c91e-aeb1-4843-a8ab-0882781d767f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:37:32.424054 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.424033 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d0c3c91e-aeb1-4843-a8ab-0882781d767f" (UID: "d0c3c91e-aeb1-4843-a8ab-0882781d767f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:37:32.424242 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.424212 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d0c3c91e-aeb1-4843-a8ab-0882781d767f" (UID: "d0c3c91e-aeb1-4843-a8ab-0882781d767f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:37:32.424339 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.424270 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0c3c91e-aeb1-4843-a8ab-0882781d767f-kube-api-access-xf69m" (OuterVolumeSpecName: "kube-api-access-xf69m") pod "d0c3c91e-aeb1-4843-a8ab-0882781d767f" (UID: "d0c3c91e-aeb1-4843-a8ab-0882781d767f"). InnerVolumeSpecName "kube-api-access-xf69m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:37:32.523656 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.523605 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-service-ca\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:37:32.523656 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.523650 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-config\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:37:32.523656 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.523659 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-oauth-serving-cert\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:37:32.523656 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.523668 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-serving-cert\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:37:32.523920 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.523677 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c3c91e-aeb1-4843-a8ab-0882781d767f-trusted-ca-bundle\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:37:32.523920 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.523685 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf69m\" (UniqueName: \"kubernetes.io/projected/d0c3c91e-aeb1-4843-a8ab-0882781d767f-kube-api-access-xf69m\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:37:32.523920 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:32.523694 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0c3c91e-aeb1-4843-a8ab-0882781d767f-console-oauth-config\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:37:33.113041 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:33.113008 2574 generic.go:356] "Generic (PLEG): container finished" podID="d0c3c91e-aeb1-4843-a8ab-0882781d767f" containerID="af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae" exitCode=0 Sep 06 10:37:33.113497 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:33.113063 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7656c95999-6pvms" Sep 06 10:37:33.113497 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:33.113101 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7656c95999-6pvms" event={"ID":"d0c3c91e-aeb1-4843-a8ab-0882781d767f","Type":"ContainerDied","Data":"af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae"} Sep 06 10:37:33.113497 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:33.113173 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7656c95999-6pvms" event={"ID":"d0c3c91e-aeb1-4843-a8ab-0882781d767f","Type":"ContainerDied","Data":"fbaccbd45b18007a8d3fc9202c450820526a1166ebb63c13428df059c6328ae6"} Sep 06 10:37:33.113497 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:33.113199 2574 scope.go:117] "RemoveContainer" containerID="af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae" Sep 06 10:37:33.121820 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:33.121804 2574 scope.go:117] "RemoveContainer" containerID="af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae" Sep 06 10:37:33.122093 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:37:33.122076 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae\": container with ID starting with af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae not found: ID does not exist" containerID="af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae" Sep 06 10:37:33.122209 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:33.122099 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae"} err="failed to get container status \"af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae\": rpc error: code = NotFound desc = could not find container \"af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae\": container with ID starting with af808a02180ea7a5bf818f8be1b6aec45ac132f7fc56b6797597c25dff79f8ae not found: ID does not exist" Sep 06 10:37:33.129261 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:33.129238 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7656c95999-6pvms"] Sep 06 10:37:33.132898 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:33.132874 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7656c95999-6pvms"] Sep 06 10:37:34.926317 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:34.926280 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0c3c91e-aeb1-4843-a8ab-0882781d767f" path="/var/lib/kubelet/pods/d0c3c91e-aeb1-4843-a8ab-0882781d767f/volumes" Sep 06 10:37:36.637399 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:36.637367 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:36.938842 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:36.938811 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:44.022388 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:44.022357 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:44.120253 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:44.120224 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:44.217171 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:44.217082 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:44.395936 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:44.395853 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:44.526093 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:44.526062 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:47.543446 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:47.543401 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:37:47.642898 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:37:47.642862 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:13.325518 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:13.325486 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:20.154462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:20.154396 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:20.251753 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:20.251719 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:21.482966 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:21.482933 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:21.583353 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:21.583313 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:21.683911 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:21.683881 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:22.031853 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.031819 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:22.133280 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.133251 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:22.209792 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.209756 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-jdfbt"] Sep 06 10:38:22.210132 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.210109 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0c3c91e-aeb1-4843-a8ab-0882781d767f" containerName="console" Sep 06 10:38:22.210181 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.210136 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0c3c91e-aeb1-4843-a8ab-0882781d767f" containerName="console" Sep 06 10:38:22.210233 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.210223 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0c3c91e-aeb1-4843-a8ab-0882781d767f" containerName="console" Sep 06 10:38:22.211926 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.211898 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:22.214383 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.214366 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" Sep 06 10:38:22.216918 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.216899 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 06 10:38:22.217019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.216905 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-n8tp6\"" Sep 06 10:38:22.225017 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.224990 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-jdfbt"] Sep 06 10:38:22.240292 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.240268 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d9b8533-457c-43a4-a96e-c42b3ad07119-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-jdfbt\" (UID: \"7d9b8533-457c-43a4-a96e-c42b3ad07119\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" Sep 06 10:38:22.240402 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.240299 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4p6v\" (UniqueName: \"kubernetes.io/projected/7d9b8533-457c-43a4-a96e-c42b3ad07119-kube-api-access-p4p6v\") pod \"payload-processing-5b54b7fbb7-jdfbt\" (UID: \"7d9b8533-457c-43a4-a96e-c42b3ad07119\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" Sep 06 10:38:22.340946 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.340872 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d9b8533-457c-43a4-a96e-c42b3ad07119-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-jdfbt\" (UID: \"7d9b8533-457c-43a4-a96e-c42b3ad07119\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" Sep 06 10:38:22.340946 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.340903 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4p6v\" (UniqueName: \"kubernetes.io/projected/7d9b8533-457c-43a4-a96e-c42b3ad07119-kube-api-access-p4p6v\") pod \"payload-processing-5b54b7fbb7-jdfbt\" (UID: \"7d9b8533-457c-43a4-a96e-c42b3ad07119\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" Sep 06 10:38:22.341613 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.341593 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d9b8533-457c-43a4-a96e-c42b3ad07119-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-jdfbt\" (UID: \"7d9b8533-457c-43a4-a96e-c42b3ad07119\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" Sep 06 10:38:22.348621 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.348601 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4p6v\" (UniqueName: \"kubernetes.io/projected/7d9b8533-457c-43a4-a96e-c42b3ad07119-kube-api-access-p4p6v\") pod \"payload-processing-5b54b7fbb7-jdfbt\" (UID: \"7d9b8533-457c-43a4-a96e-c42b3ad07119\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" Sep 06 10:38:22.525628 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.525591 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" Sep 06 10:38:22.797282 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:22.797254 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-jdfbt"] Sep 06 10:38:22.798913 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:38:22.798883 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d9b8533_457c_43a4_a96e_c42b3ad07119.slice/crio-27311d7a307b453d6ee49b74c2d8b523f7c570320285b1be33b1f426e9797cc7 WatchSource:0}: Error finding container 27311d7a307b453d6ee49b74c2d8b523f7c570320285b1be33b1f426e9797cc7: Status 404 returned error can't find the container with id 27311d7a307b453d6ee49b74c2d8b523f7c570320285b1be33b1f426e9797cc7 Sep 06 10:38:23.380467 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:23.380426 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" event={"ID":"7d9b8533-457c-43a4-a96e-c42b3ad07119","Type":"ContainerStarted","Data":"27311d7a307b453d6ee49b74c2d8b523f7c570320285b1be33b1f426e9797cc7"} Sep 06 10:38:25.392226 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:25.392188 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" event={"ID":"7d9b8533-457c-43a4-a96e-c42b3ad07119","Type":"ContainerStarted","Data":"a008f58f147055f56e67aa8d3ead50a7d15e181df4d718741dc1c02dc72921c5"} Sep 06 10:38:25.392631 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:25.392327 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" Sep 06 10:38:25.409457 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:25.409404 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" podStartSLOduration=1.802392889 podStartE2EDuration="3.409390009s" totalImagesPullingTime="1.60699712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:38:25.408142059 +0000 UTC m=+498.990631690" watchObservedRunningTime="2026-09-06 10:38:25.409390009 +0000 UTC m=+498.991879639" Sep 06 10:38:31.555337 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:31.555299 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:31.657732 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:31.657702 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:36.398634 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:36.398601 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-jdfbt" Sep 06 10:38:38.237597 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.237560 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg"] Sep 06 10:38:38.241512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.241491 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.244926 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.244897 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 06 10:38:38.244926 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.244920 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 06 10:38:38.245139 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.244974 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-h68bt\"" Sep 06 10:38:38.245139 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.245001 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 06 10:38:38.249670 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.249649 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg"] Sep 06 10:38:38.268340 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.268308 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:38.372308 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.372278 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/836e23c7-15f2-4613-a54c-16e02646aadc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.372468 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.372319 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcbcz\" (UniqueName: \"kubernetes.io/projected/836e23c7-15f2-4613-a54c-16e02646aadc-kube-api-access-pcbcz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.372468 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.372345 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/836e23c7-15f2-4613-a54c-16e02646aadc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.372468 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.372422 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/836e23c7-15f2-4613-a54c-16e02646aadc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.372634 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.372467 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/836e23c7-15f2-4613-a54c-16e02646aadc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.372634 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.372532 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/836e23c7-15f2-4613-a54c-16e02646aadc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.472984 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.472953 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/836e23c7-15f2-4613-a54c-16e02646aadc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.473191 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.472995 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/836e23c7-15f2-4613-a54c-16e02646aadc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.473191 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.473118 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/836e23c7-15f2-4613-a54c-16e02646aadc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.473285 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.473192 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcbcz\" (UniqueName: \"kubernetes.io/projected/836e23c7-15f2-4613-a54c-16e02646aadc-kube-api-access-pcbcz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.473285 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.473222 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/836e23c7-15f2-4613-a54c-16e02646aadc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.473285 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.473258 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/836e23c7-15f2-4613-a54c-16e02646aadc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.473481 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.473456 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/836e23c7-15f2-4613-a54c-16e02646aadc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.473594 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.473544 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/836e23c7-15f2-4613-a54c-16e02646aadc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.473654 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.473593 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/836e23c7-15f2-4613-a54c-16e02646aadc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.475737 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.475713 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/836e23c7-15f2-4613-a54c-16e02646aadc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.476111 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.476087 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/836e23c7-15f2-4613-a54c-16e02646aadc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.481247 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.481228 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcbcz\" (UniqueName: \"kubernetes.io/projected/836e23c7-15f2-4613-a54c-16e02646aadc-kube-api-access-pcbcz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg\" (UID: \"836e23c7-15f2-4613-a54c-16e02646aadc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.553392 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.553328 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:38.824749 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:38.824721 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg"] Sep 06 10:38:38.827044 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:38:38.827006 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod836e23c7_15f2_4613_a54c_16e02646aadc.slice/crio-1cf304198550cdfae26d7b40f4281ccc4a4eaf32cd382b3d7de9311a32971075 WatchSource:0}: Error finding container 1cf304198550cdfae26d7b40f4281ccc4a4eaf32cd382b3d7de9311a32971075: Status 404 returned error can't find the container with id 1cf304198550cdfae26d7b40f4281ccc4a4eaf32cd382b3d7de9311a32971075 Sep 06 10:38:39.465933 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:39.465893 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" event={"ID":"836e23c7-15f2-4613-a54c-16e02646aadc","Type":"ContainerStarted","Data":"1cf304198550cdfae26d7b40f4281ccc4a4eaf32cd382b3d7de9311a32971075"} Sep 06 10:38:40.473052 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:40.473013 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" event={"ID":"836e23c7-15f2-4613-a54c-16e02646aadc","Type":"ContainerStarted","Data":"752711a903fd6d115d2f901f0e03a75e371e06dbbfb9e6e04af8f010dd171380"} Sep 06 10:38:40.493885 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:40.493831 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" podStartSLOduration=1.442133922 podStartE2EDuration="2.493816326s" totalImagesPullingTime="1.051682404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:38:40.491016394 +0000 UTC m=+514.073506026" watchObservedRunningTime="2026-09-06 10:38:40.493816326 +0000 UTC m=+514.076305956" Sep 06 10:38:47.819726 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:47.819691 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:38:48.553769 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:48.553738 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:48.553940 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:48.553831 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:48.566554 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:48.566530 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:49.528251 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:49.528225 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg" Sep 06 10:38:53.837581 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:38:53.837545 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:39:05.595670 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:05.595589 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:39:05.708834 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:05.708796 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:39:15.238012 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.237981 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc"] Sep 06 10:39:15.241627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.241606 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.243863 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.243843 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 06 10:39:15.252370 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.252344 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc"] Sep 06 10:39:15.259722 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.259701 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:39:15.311168 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.311113 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.311168 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.311169 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.311358 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.311217 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.311358 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.311235 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.311358 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.311280 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.311358 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.311297 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5wcp\" (UniqueName: \"kubernetes.io/projected/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-kube-api-access-p5wcp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.412191 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.412150 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.412191 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.412190 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.412391 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.412216 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.412391 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.412268 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.412391 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.412317 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.412391 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.412337 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5wcp\" (UniqueName: \"kubernetes.io/projected/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-kube-api-access-p5wcp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.412580 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.412557 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.412679 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.412657 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.412782 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.412760 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.414461 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.414443 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.414563 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.414547 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.423650 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.423625 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5wcp\" (UniqueName: \"kubernetes.io/projected/03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815-kube-api-access-p5wcp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-92sxc\" (UID: \"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.553514 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.553424 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:15.827258 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:15.827232 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc"] Sep 06 10:39:15.828888 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:39:15.828866 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d1a85a_49f6_4eb8_9e9f_30cdf8a2e815.slice/crio-8538ca9d86c840119ce933e520deca301d3136cff147b8c24862614c15254456 WatchSource:0}: Error finding container 8538ca9d86c840119ce933e520deca301d3136cff147b8c24862614c15254456: Status 404 returned error can't find the container with id 8538ca9d86c840119ce933e520deca301d3136cff147b8c24862614c15254456 Sep 06 10:39:16.655682 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:16.655646 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" event={"ID":"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815","Type":"ContainerStarted","Data":"08e2ad7e68ad573ddfd799ccdd013359905bca60703b245e243fa4a8060af2ec"} Sep 06 10:39:16.656052 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:16.655696 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" event={"ID":"03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815","Type":"ContainerStarted","Data":"8538ca9d86c840119ce933e520deca301d3136cff147b8c24862614c15254456"} Sep 06 10:39:16.674946 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:16.674895 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" podStartSLOduration=1.513123222 podStartE2EDuration="1.674883271s" totalImagesPullingTime="161.760049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:39:16.672086274 +0000 UTC m=+550.254575905" watchObservedRunningTime="2026-09-06 10:39:16.674883271 +0000 UTC m=+550.257372901" Sep 06 10:39:16.725428 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:16.725400 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:39:16.826508 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:16.826476 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:39:21.755194 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:21.755158 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:39:25.553821 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:25.553770 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:25.554273 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:25.553860 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:25.566235 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:25.566207 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:25.711029 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:25.710996 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-92sxc" Sep 06 10:39:28.750556 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:28.750520 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:39:33.949387 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:39:33.949350 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:40:04.962560 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:40:04.962493 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:40:44.726446 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:40:44.726401 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:40:44.821815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:40:44.821781 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:40:45.812392 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:40:45.812355 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:40:45.922213 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:40:45.922176 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:00.899865 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:00.899832 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:01.012787 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.012751 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:01.214517 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.214487 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz"] Sep 06 10:41:01.217861 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.217842 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.224801 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.224770 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-232aeb-openshift-default-dockercfg-9q8n9\"" Sep 06 10:41:01.231243 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.231211 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz"] Sep 06 10:41:01.232772 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.232714 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:01.326032 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.326003 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-envoy\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.326032 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.326033 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-podinfo\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.326276 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.326054 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-workload-socket\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.326276 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.326068 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-data\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.326276 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.326088 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-token\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.326276 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.326170 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-credential-socket\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.326276 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.326225 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-workload-certs\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.326276 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.326278 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istiod-ca-cert\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.326493 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.326307 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drlhr\" (UniqueName: \"kubernetes.io/projected/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-kube-api-access-drlhr\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.426935 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.426894 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-envoy\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.426935 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.426934 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-podinfo\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.427232 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.426953 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-workload-socket\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.427232 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.426970 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-data\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.427232 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.427013 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-token\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.427232 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.427048 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-credential-socket\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.427443 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.427290 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-workload-certs\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.427443 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.427384 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istiod-ca-cert\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.427443 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.427407 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-workload-socket\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.427443 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.427415 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drlhr\" (UniqueName: \"kubernetes.io/projected/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-kube-api-access-drlhr\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.427724 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.427517 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-credential-socket\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.427724 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.427626 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-workload-certs\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.427836 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.427774 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-data\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.428119 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.428092 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istiod-ca-cert\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.429415 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.429394 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-envoy\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.429649 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.429632 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-podinfo\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.437353 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.437331 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-token\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.440611 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.440593 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drlhr\" (UniqueName: \"kubernetes.io/projected/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-kube-api-access-drlhr\") pod \"e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.537167 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.537043 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:01.684872 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.684838 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:01.811106 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.811078 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz"] Sep 06 10:41:01.812794 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:41:01.812767 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad17d760_53d2_4cb2_afb7_e6eb810a4b18.slice/crio-d3f560573f9decd06c25b7c7dae3f4e234b143ac2e4d3fc9ab5ebada85fb93fc WatchSource:0}: Error finding container d3f560573f9decd06c25b7c7dae3f4e234b143ac2e4d3fc9ab5ebada85fb93fc: Status 404 returned error can't find the container with id d3f560573f9decd06c25b7c7dae3f4e234b143ac2e4d3fc9ab5ebada85fb93fc Sep 06 10:41:01.814857 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.814828 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:41:01.814915 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.814892 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:41:01.814952 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:01.814920 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:41:02.208765 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:02.208723 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" event={"ID":"ad17d760-53d2-4cb2-afb7-e6eb810a4b18","Type":"ContainerStarted","Data":"492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503"} Sep 06 10:41:02.209218 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:02.208782 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" event={"ID":"ad17d760-53d2-4cb2-afb7-e6eb810a4b18","Type":"ContainerStarted","Data":"d3f560573f9decd06c25b7c7dae3f4e234b143ac2e4d3fc9ab5ebada85fb93fc"} Sep 06 10:41:02.238870 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:02.238817 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" podStartSLOduration=1.238799588 podStartE2EDuration="1.238799588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:02.236029521 +0000 UTC m=+655.818519179" watchObservedRunningTime="2026-09-06 10:41:02.238799588 +0000 UTC m=+655.821289218" Sep 06 10:41:02.277187 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:02.277116 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:02.377719 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:02.377686 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:02.538105 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:02.538029 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:02.539520 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:02.539490 2574 patch_prober.go:28] interesting pod/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Sep 06 10:41:02.539647 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:02.539540 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Sep 06 10:41:02.708180 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:02.708140 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:03.541267 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:03.541239 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:04.161382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:04.161324 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:04.220734 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:04.220706 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:04.221658 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:04.221640 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:04.251309 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:04.251280 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:05.309274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:05.309228 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:05.411033 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:05.410984 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:05.619381 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:05.619269 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:05.901628 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:05.901531 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:06.004431 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.004399 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:06.012405 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.012373 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz"] Sep 06 10:41:06.015654 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.015639 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" Sep 06 10:41:06.017963 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.017935 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-232aeb\"" Sep 06 10:41:06.018065 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.017968 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-232aeb-dockercfg-wcqlp\"" Sep 06 10:41:06.026578 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.026554 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz"] Sep 06 10:41:06.054100 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.054065 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr"] Sep 06 10:41:06.057468 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.057452 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" Sep 06 10:41:06.065241 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.065216 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr"] Sep 06 10:41:06.067778 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.067753 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw2nj\" (UniqueName: \"kubernetes.io/projected/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3-kube-api-access-mw2nj\") pod \"payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz\" (UID: \"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" Sep 06 10:41:06.067881 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.067803 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz\" (UID: \"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" Sep 06 10:41:06.168470 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.168437 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz\" (UID: \"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" Sep 06 10:41:06.168656 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.168493 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86dae43f-0771-497d-bd80-ee09793cfe39-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr\" (UID: \"86dae43f-0771-497d-bd80-ee09793cfe39\") " pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" Sep 06 10:41:06.168656 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.168547 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw2nj\" (UniqueName: \"kubernetes.io/projected/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3-kube-api-access-mw2nj\") pod \"payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz\" (UID: \"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" Sep 06 10:41:06.168656 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.168567 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtcw4\" (UniqueName: \"kubernetes.io/projected/86dae43f-0771-497d-bd80-ee09793cfe39-kube-api-access-rtcw4\") pod \"payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr\" (UID: \"86dae43f-0771-497d-bd80-ee09793cfe39\") " pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" Sep 06 10:41:06.169282 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.169260 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz\" (UID: \"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" Sep 06 10:41:06.176538 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.176516 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw2nj\" (UniqueName: \"kubernetes.io/projected/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3-kube-api-access-mw2nj\") pod \"payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz\" (UID: \"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" Sep 06 10:41:06.269948 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.269915 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtcw4\" (UniqueName: \"kubernetes.io/projected/86dae43f-0771-497d-bd80-ee09793cfe39-kube-api-access-rtcw4\") pod \"payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr\" (UID: \"86dae43f-0771-497d-bd80-ee09793cfe39\") " pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" Sep 06 10:41:06.270159 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.269989 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86dae43f-0771-497d-bd80-ee09793cfe39-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr\" (UID: \"86dae43f-0771-497d-bd80-ee09793cfe39\") " pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" Sep 06 10:41:06.270665 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.270646 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86dae43f-0771-497d-bd80-ee09793cfe39-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr\" (UID: \"86dae43f-0771-497d-bd80-ee09793cfe39\") " pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" Sep 06 10:41:06.277320 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.277280 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtcw4\" (UniqueName: \"kubernetes.io/projected/86dae43f-0771-497d-bd80-ee09793cfe39-kube-api-access-rtcw4\") pod \"payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr\" (UID: \"86dae43f-0771-497d-bd80-ee09793cfe39\") " pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" Sep 06 10:41:06.327088 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.327052 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" Sep 06 10:41:06.368603 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.368566 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" Sep 06 10:41:06.598473 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.598449 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz"] Sep 06 10:41:06.600346 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:41:06.600316 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3dd3055_ccf9_4ba6_99bb_9e7003fa6ed3.slice/crio-466f6542918ff0774838f59a637d2995d6005d3980dcf4b20b7a89a99459de5b WatchSource:0}: Error finding container 466f6542918ff0774838f59a637d2995d6005d3980dcf4b20b7a89a99459de5b: Status 404 returned error can't find the container with id 466f6542918ff0774838f59a637d2995d6005d3980dcf4b20b7a89a99459de5b Sep 06 10:41:06.653655 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:06.653606 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr"] Sep 06 10:41:06.655463 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:41:06.655441 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86dae43f_0771_497d_bd80_ee09793cfe39.slice/crio-fa06430e8f3014b4092f107fc33e77840e8d5c0835b776366f7437011df27ce0 WatchSource:0}: Error finding container fa06430e8f3014b4092f107fc33e77840e8d5c0835b776366f7437011df27ce0: Status 404 returned error can't find the container with id fa06430e8f3014b4092f107fc33e77840e8d5c0835b776366f7437011df27ce0 Sep 06 10:41:07.239317 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:07.239284 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" event={"ID":"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3","Type":"ContainerStarted","Data":"9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9"} Sep 06 10:41:07.239861 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:07.239343 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" event={"ID":"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3","Type":"ContainerStarted","Data":"466f6542918ff0774838f59a637d2995d6005d3980dcf4b20b7a89a99459de5b"} Sep 06 10:41:07.239861 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:07.239390 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" Sep 06 10:41:07.241089 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:07.241064 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" event={"ID":"86dae43f-0771-497d-bd80-ee09793cfe39","Type":"ContainerStarted","Data":"e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3"} Sep 06 10:41:07.241220 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:07.241110 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" event={"ID":"86dae43f-0771-497d-bd80-ee09793cfe39","Type":"ContainerStarted","Data":"fa06430e8f3014b4092f107fc33e77840e8d5c0835b776366f7437011df27ce0"} Sep 06 10:41:07.241220 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:07.241168 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" Sep 06 10:41:07.260638 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:07.260589 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" podStartSLOduration=2.260573232 podStartE2EDuration="2.260573232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:07.25685887 +0000 UTC m=+660.839348506" watchObservedRunningTime="2026-09-06 10:41:07.260573232 +0000 UTC m=+660.843062864" Sep 06 10:41:07.275446 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:07.275401 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" podStartSLOduration=1.275387033 podStartE2EDuration="1.275387033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:07.273669271 +0000 UTC m=+660.856158901" watchObservedRunningTime="2026-09-06 10:41:07.275387033 +0000 UTC m=+660.857876666" Sep 06 10:41:08.675229 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:08.675193 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:08.730076 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:08.730045 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:08.892338 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:08.892303 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:09.219954 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:09.219917 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:09.386859 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:09.386811 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:10.095107 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:10.095076 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:10.466751 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:10.466720 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:10.706507 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:10.706474 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:11.817690 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:11.817647 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:11.902366 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:11.902333 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:13.069752 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:13.069721 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:13.195996 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:13.195965 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:13.412914 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:13.412836 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:13.521020 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:13.520989 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:13.844147 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:13.840397 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:13.936411 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:13.936116 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:14.020841 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:14.020799 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc"] Sep 06 10:41:14.024665 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:14.024642 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" Sep 06 10:41:14.028263 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:14.028236 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e0690faf\"" Sep 06 10:41:14.043054 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:14.043020 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc"] Sep 06 10:41:14.146742 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:14.146642 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99945164-d23b-4ade-a6d2-3707c354c057-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc\" (UID: \"99945164-d23b-4ade-a6d2-3707c354c057\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" Sep 06 10:41:14.146742 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:14.146702 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6kmn\" (UniqueName: \"kubernetes.io/projected/99945164-d23b-4ade-a6d2-3707c354c057-kube-api-access-t6kmn\") pod \"payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc\" (UID: \"99945164-d23b-4ade-a6d2-3707c354c057\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" Sep 06 10:41:14.247834 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:14.247789 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99945164-d23b-4ade-a6d2-3707c354c057-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc\" (UID: \"99945164-d23b-4ade-a6d2-3707c354c057\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" Sep 06 10:41:14.248081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:14.248060 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6kmn\" (UniqueName: \"kubernetes.io/projected/99945164-d23b-4ade-a6d2-3707c354c057-kube-api-access-t6kmn\") pod \"payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc\" (UID: \"99945164-d23b-4ade-a6d2-3707c354c057\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" Sep 06 10:41:14.249215 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:14.249185 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99945164-d23b-4ade-a6d2-3707c354c057-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc\" (UID: \"99945164-d23b-4ade-a6d2-3707c354c057\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" Sep 06 10:41:14.258292 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:14.258265 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6kmn\" (UniqueName: \"kubernetes.io/projected/99945164-d23b-4ade-a6d2-3707c354c057-kube-api-access-t6kmn\") pod \"payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc\" (UID: \"99945164-d23b-4ade-a6d2-3707c354c057\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" Sep 06 10:41:14.337783 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:14.337734 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" Sep 06 10:41:14.640064 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:14.640029 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc"] Sep 06 10:41:14.653731 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:41:14.653496 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99945164_d23b_4ade_a6d2_3707c354c057.slice/crio-07a01072da5cb3de1eb35e29ba1e3c89ad93133715fef5e6bafc5012b662e518 WatchSource:0}: Error finding container 07a01072da5cb3de1eb35e29ba1e3c89ad93133715fef5e6bafc5012b662e518: Status 404 returned error can't find the container with id 07a01072da5cb3de1eb35e29ba1e3c89ad93133715fef5e6bafc5012b662e518 Sep 06 10:41:15.221057 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.221007 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:15.288981 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.288930 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" event={"ID":"99945164-d23b-4ade-a6d2-3707c354c057","Type":"ContainerStarted","Data":"2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688"} Sep 06 10:41:15.289197 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.288995 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" event={"ID":"99945164-d23b-4ade-a6d2-3707c354c057","Type":"ContainerStarted","Data":"07a01072da5cb3de1eb35e29ba1e3c89ad93133715fef5e6bafc5012b662e518"} Sep 06 10:41:15.289197 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.289037 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" Sep 06 10:41:15.289823 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.289791 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:15.307210 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.307158 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" podStartSLOduration=2.307144107 podStartE2EDuration="2.307144107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:15.304907731 +0000 UTC m=+668.887397364" watchObservedRunningTime="2026-09-06 10:41:15.307144107 +0000 UTC m=+668.889633759" Sep 06 10:41:15.390541 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.390507 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:15.764946 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.764909 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:15.851853 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.851823 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:15.902372 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.902337 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg"] Sep 06 10:41:15.906023 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.906006 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" Sep 06 10:41:15.908301 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.908282 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-55111c\"" Sep 06 10:41:15.915179 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.915154 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg"] Sep 06 10:41:15.965367 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.965329 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487"] Sep 06 10:41:15.969603 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.969579 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" Sep 06 10:41:15.971929 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.971907 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-55111c-dockercfg-m9lfk\"" Sep 06 10:41:15.977575 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:15.977408 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487"] Sep 06 10:41:16.068449 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.068360 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndsth\" (UniqueName: \"kubernetes.io/projected/97e8df12-9ca3-4b1e-804d-88e9812c9417-kube-api-access-ndsth\") pod \"payload-processing-e2e-autores-55111c-bc5896cc-cj2lg\" (UID: \"97e8df12-9ca3-4b1e-804d-88e9812c9417\") " pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" Sep 06 10:41:16.068449 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.068398 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/166d191f-bb72-4ff6-b12d-f58873442f1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487\" (UID: \"166d191f-bb72-4ff6-b12d-f58873442f1e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" Sep 06 10:41:16.068449 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.068424 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e8df12-9ca3-4b1e-804d-88e9812c9417-plugins-config-volume\") pod \"payload-processing-e2e-autores-55111c-bc5896cc-cj2lg\" (UID: \"97e8df12-9ca3-4b1e-804d-88e9812c9417\") " pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" Sep 06 10:41:16.068663 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.068568 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k956f\" (UniqueName: \"kubernetes.io/projected/166d191f-bb72-4ff6-b12d-f58873442f1e-kube-api-access-k956f\") pod \"payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487\" (UID: \"166d191f-bb72-4ff6-b12d-f58873442f1e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" Sep 06 10:41:16.169210 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.169171 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndsth\" (UniqueName: \"kubernetes.io/projected/97e8df12-9ca3-4b1e-804d-88e9812c9417-kube-api-access-ndsth\") pod \"payload-processing-e2e-autores-55111c-bc5896cc-cj2lg\" (UID: \"97e8df12-9ca3-4b1e-804d-88e9812c9417\") " pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" Sep 06 10:41:16.169431 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.169221 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/166d191f-bb72-4ff6-b12d-f58873442f1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487\" (UID: \"166d191f-bb72-4ff6-b12d-f58873442f1e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" Sep 06 10:41:16.169431 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.169262 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e8df12-9ca3-4b1e-804d-88e9812c9417-plugins-config-volume\") pod \"payload-processing-e2e-autores-55111c-bc5896cc-cj2lg\" (UID: \"97e8df12-9ca3-4b1e-804d-88e9812c9417\") " pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" Sep 06 10:41:16.169431 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.169356 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k956f\" (UniqueName: \"kubernetes.io/projected/166d191f-bb72-4ff6-b12d-f58873442f1e-kube-api-access-k956f\") pod \"payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487\" (UID: \"166d191f-bb72-4ff6-b12d-f58873442f1e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" Sep 06 10:41:16.169984 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.169959 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/166d191f-bb72-4ff6-b12d-f58873442f1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487\" (UID: \"166d191f-bb72-4ff6-b12d-f58873442f1e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" Sep 06 10:41:16.170082 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.170063 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e8df12-9ca3-4b1e-804d-88e9812c9417-plugins-config-volume\") pod \"payload-processing-e2e-autores-55111c-bc5896cc-cj2lg\" (UID: \"97e8df12-9ca3-4b1e-804d-88e9812c9417\") " pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" Sep 06 10:41:16.177702 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.177679 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k956f\" (UniqueName: \"kubernetes.io/projected/166d191f-bb72-4ff6-b12d-f58873442f1e-kube-api-access-k956f\") pod \"payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487\" (UID: \"166d191f-bb72-4ff6-b12d-f58873442f1e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" Sep 06 10:41:16.177831 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.177715 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndsth\" (UniqueName: \"kubernetes.io/projected/97e8df12-9ca3-4b1e-804d-88e9812c9417-kube-api-access-ndsth\") pod \"payload-processing-e2e-autores-55111c-bc5896cc-cj2lg\" (UID: \"97e8df12-9ca3-4b1e-804d-88e9812c9417\") " pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" Sep 06 10:41:16.218202 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.218169 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" Sep 06 10:41:16.281888 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.281855 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" Sep 06 10:41:16.515612 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.515584 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg"] Sep 06 10:41:16.516993 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:41:16.516965 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e8df12_9ca3_4b1e_804d_88e9812c9417.slice/crio-bdada11f65b36fc34930b85893657bf47bf6a4149dee29acdddab4316cf6a913 WatchSource:0}: Error finding container bdada11f65b36fc34930b85893657bf47bf6a4149dee29acdddab4316cf6a913: Status 404 returned error can't find the container with id bdada11f65b36fc34930b85893657bf47bf6a4149dee29acdddab4316cf6a913 Sep 06 10:41:16.600054 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:16.600028 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487"] Sep 06 10:41:16.600980 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:41:16.600952 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod166d191f_bb72_4ff6_b12d_f58873442f1e.slice/crio-be10f6e2ef40608ec02133f875b039028b5e8328e7dba9d190882e7c257b0b39 WatchSource:0}: Error finding container be10f6e2ef40608ec02133f875b039028b5e8328e7dba9d190882e7c257b0b39: Status 404 returned error can't find the container with id be10f6e2ef40608ec02133f875b039028b5e8328e7dba9d190882e7c257b0b39 Sep 06 10:41:17.306413 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:17.306377 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" event={"ID":"166d191f-bb72-4ff6-b12d-f58873442f1e","Type":"ContainerStarted","Data":"60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4"} Sep 06 10:41:17.306935 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:17.306454 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" event={"ID":"166d191f-bb72-4ff6-b12d-f58873442f1e","Type":"ContainerStarted","Data":"be10f6e2ef40608ec02133f875b039028b5e8328e7dba9d190882e7c257b0b39"} Sep 06 10:41:17.306935 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:17.306488 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" Sep 06 10:41:17.308071 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:17.308047 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" event={"ID":"97e8df12-9ca3-4b1e-804d-88e9812c9417","Type":"ContainerStarted","Data":"44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b"} Sep 06 10:41:17.308212 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:17.308087 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" event={"ID":"97e8df12-9ca3-4b1e-804d-88e9812c9417","Type":"ContainerStarted","Data":"bdada11f65b36fc34930b85893657bf47bf6a4149dee29acdddab4316cf6a913"} Sep 06 10:41:17.308212 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:17.308118 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" Sep 06 10:41:17.330310 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:17.330253 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" podStartSLOduration=2.330235112 podStartE2EDuration="2.330235112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:17.327881185 +0000 UTC m=+670.910370816" watchObservedRunningTime="2026-09-06 10:41:17.330235112 +0000 UTC m=+670.912724750" Sep 06 10:41:17.357652 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:17.357575 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" podStartSLOduration=2.357535515 podStartE2EDuration="2.357535515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:17.349152952 +0000 UTC m=+670.931642584" watchObservedRunningTime="2026-09-06 10:41:17.357535515 +0000 UTC m=+670.940025139" Sep 06 10:41:18.248984 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:18.248947 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" Sep 06 10:41:18.249807 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:18.249778 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" Sep 06 10:41:19.652409 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:19.652330 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:19.801494 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:19.801466 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:19.906274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:19.906193 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:20.001160 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:20.001096 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:22.343729 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:22.343698 2574 generic.go:356] "Generic (PLEG): container finished" podID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerID="492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503" exitCode=0 Sep 06 10:41:22.344165 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:22.343735 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" event={"ID":"ad17d760-53d2-4cb2-afb7-e6eb810a4b18","Type":"ContainerDied","Data":"492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503"} Sep 06 10:41:22.344165 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:22.344117 2574 scope.go:117] "RemoveContainer" containerID="492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503" Sep 06 10:41:22.345455 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:22.345425 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:41:22.345514 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:22.345495 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:41:22.345549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:22.345528 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:41:22.682870 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:22.682833 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:23.355292 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:23.355259 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" event={"ID":"ad17d760-53d2-4cb2-afb7-e6eb810a4b18","Type":"ContainerStarted","Data":"0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802"} Sep 06 10:41:23.538345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:23.538299 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:23.538561 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:23.538443 2574 patch_prober.go:28] interesting pod/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Sep 06 10:41:23.538561 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:23.538487 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Sep 06 10:41:24.254560 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:24.254516 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:24.538531 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:24.538426 2574 patch_prober.go:28] interesting pod/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Sep 06 10:41:24.538531 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:24.538502 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Sep 06 10:41:25.538381 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:25.538331 2574 patch_prober.go:28] interesting pod/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Sep 06 10:41:25.538598 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:25.538405 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Sep 06 10:41:26.301551 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:26.301523 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" Sep 06 10:41:26.538168 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:26.538099 2574 patch_prober.go:28] interesting pod/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Sep 06 10:41:26.538376 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:26.538194 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Sep 06 10:41:27.350836 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:27.350796 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:27.538143 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:27.538078 2574 patch_prober.go:28] interesting pod/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Sep 06 10:41:27.538429 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:27.538400 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Sep 06 10:41:28.317626 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:28.317593 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" Sep 06 10:41:28.317787 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:28.317671 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" Sep 06 10:41:28.537489 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:28.537449 2574 patch_prober.go:28] interesting pod/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Sep 06 10:41:28.537859 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:28.537509 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Sep 06 10:41:29.099107 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:29.099071 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:29.415697 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:29.415661 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:29.541960 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:29.541928 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:29.542412 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:29.542192 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:29.542950 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:29.542932 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:41:29.753519 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:29.753440 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:29.947535 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:29.947497 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:32.433666 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:32.433634 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:32.535442 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:32.535413 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:33.591975 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:33.591944 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:33.695895 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:33.695863 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:33.801902 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:33.801867 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:34.066863 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.066832 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:34.167892 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.167857 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:34.188274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.188240 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg"] Sep 06 10:41:34.191943 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.191922 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" Sep 06 10:41:34.194299 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.194274 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-636b04\"" Sep 06 10:41:34.194417 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.194314 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-636b04-dockercfg-zd7nf\"" Sep 06 10:41:34.199846 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.199827 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc"] Sep 06 10:41:34.203060 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.203040 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" Sep 06 10:41:34.204732 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.204713 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg"] Sep 06 10:41:34.227145 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.227099 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc"] Sep 06 10:41:34.228340 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.228318 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ab2ec7-331b-4b12-8aba-dce4ce479db0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc\" (UID: \"b5ab2ec7-331b-4b12-8aba-dce4ce479db0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" Sep 06 10:41:34.228432 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.228345 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8xh2\" (UniqueName: \"kubernetes.io/projected/b5ab2ec7-331b-4b12-8aba-dce4ce479db0-kube-api-access-t8xh2\") pod \"payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc\" (UID: \"b5ab2ec7-331b-4b12-8aba-dce4ce479db0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" Sep 06 10:41:34.228432 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.228391 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c5e4dcd-acee-424f-b57c-45c5e547f70e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg\" (UID: \"3c5e4dcd-acee-424f-b57c-45c5e547f70e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" Sep 06 10:41:34.228515 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.228482 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz2ch\" (UniqueName: \"kubernetes.io/projected/3c5e4dcd-acee-424f-b57c-45c5e547f70e-kube-api-access-pz2ch\") pod \"payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg\" (UID: \"3c5e4dcd-acee-424f-b57c-45c5e547f70e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" Sep 06 10:41:34.329976 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.329896 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ab2ec7-331b-4b12-8aba-dce4ce479db0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc\" (UID: \"b5ab2ec7-331b-4b12-8aba-dce4ce479db0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" Sep 06 10:41:34.329976 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.329932 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8xh2\" (UniqueName: \"kubernetes.io/projected/b5ab2ec7-331b-4b12-8aba-dce4ce479db0-kube-api-access-t8xh2\") pod \"payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc\" (UID: \"b5ab2ec7-331b-4b12-8aba-dce4ce479db0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" Sep 06 10:41:34.330205 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.330070 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c5e4dcd-acee-424f-b57c-45c5e547f70e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg\" (UID: \"3c5e4dcd-acee-424f-b57c-45c5e547f70e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" Sep 06 10:41:34.330205 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.330181 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz2ch\" (UniqueName: \"kubernetes.io/projected/3c5e4dcd-acee-424f-b57c-45c5e547f70e-kube-api-access-pz2ch\") pod \"payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg\" (UID: \"3c5e4dcd-acee-424f-b57c-45c5e547f70e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" Sep 06 10:41:34.330756 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.330736 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ab2ec7-331b-4b12-8aba-dce4ce479db0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc\" (UID: \"b5ab2ec7-331b-4b12-8aba-dce4ce479db0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" Sep 06 10:41:34.330845 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.330826 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c5e4dcd-acee-424f-b57c-45c5e547f70e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg\" (UID: \"3c5e4dcd-acee-424f-b57c-45c5e547f70e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" Sep 06 10:41:34.339538 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.339503 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz2ch\" (UniqueName: \"kubernetes.io/projected/3c5e4dcd-acee-424f-b57c-45c5e547f70e-kube-api-access-pz2ch\") pod \"payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg\" (UID: \"3c5e4dcd-acee-424f-b57c-45c5e547f70e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" Sep 06 10:41:34.339673 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.339657 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8xh2\" (UniqueName: \"kubernetes.io/projected/b5ab2ec7-331b-4b12-8aba-dce4ce479db0-kube-api-access-t8xh2\") pod \"payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc\" (UID: \"b5ab2ec7-331b-4b12-8aba-dce4ce479db0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" Sep 06 10:41:34.467320 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.467288 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:34.504218 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.504179 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" Sep 06 10:41:34.512827 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.512803 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" Sep 06 10:41:34.785763 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.785739 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg"] Sep 06 10:41:34.788102 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:41:34.788074 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c5e4dcd_acee_424f_b57c_45c5e547f70e.slice/crio-f98a4258eece0ad28ce249458840d2a1616bd35969f6a8464d0ab52f63f94f7c WatchSource:0}: Error finding container f98a4258eece0ad28ce249458840d2a1616bd35969f6a8464d0ab52f63f94f7c: Status 404 returned error can't find the container with id f98a4258eece0ad28ce249458840d2a1616bd35969f6a8464d0ab52f63f94f7c Sep 06 10:41:34.799734 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:34.799263 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc"] Sep 06 10:41:34.801488 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:41:34.801458 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5ab2ec7_331b_4b12_8aba_dce4ce479db0.slice/crio-38ead67a6101240db2c2b78295df73c0e406b458ff57d66cde899aa3affd506e WatchSource:0}: Error finding container 38ead67a6101240db2c2b78295df73c0e406b458ff57d66cde899aa3affd506e: Status 404 returned error can't find the container with id 38ead67a6101240db2c2b78295df73c0e406b458ff57d66cde899aa3affd506e Sep 06 10:41:35.257065 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:35.257032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:35.425073 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:35.425031 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" event={"ID":"3c5e4dcd-acee-424f-b57c-45c5e547f70e","Type":"ContainerStarted","Data":"765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4"} Sep 06 10:41:35.425283 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:35.425105 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" event={"ID":"3c5e4dcd-acee-424f-b57c-45c5e547f70e","Type":"ContainerStarted","Data":"f98a4258eece0ad28ce249458840d2a1616bd35969f6a8464d0ab52f63f94f7c"} Sep 06 10:41:35.425283 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:35.425167 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" Sep 06 10:41:35.426749 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:35.426723 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" event={"ID":"b5ab2ec7-331b-4b12-8aba-dce4ce479db0","Type":"ContainerStarted","Data":"52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294"} Sep 06 10:41:35.426881 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:35.426763 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" event={"ID":"b5ab2ec7-331b-4b12-8aba-dce4ce479db0","Type":"ContainerStarted","Data":"38ead67a6101240db2c2b78295df73c0e406b458ff57d66cde899aa3affd506e"} Sep 06 10:41:35.426881 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:35.426797 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" Sep 06 10:41:35.447414 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:35.447361 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" podStartSLOduration=1.447345009 podStartE2EDuration="1.447345009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:35.443037368 +0000 UTC m=+689.025527000" watchObservedRunningTime="2026-09-06 10:41:35.447345009 +0000 UTC m=+689.029834640" Sep 06 10:41:35.465606 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:35.465545 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" podStartSLOduration=1.4655171089999999 podStartE2EDuration="1.465517109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:41:35.462067385 +0000 UTC m=+689.044557017" watchObservedRunningTime="2026-09-06 10:41:35.465517109 +0000 UTC m=+689.048006742" Sep 06 10:41:35.819002 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:35.818940 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:35.967929 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:35.967891 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:36.763609 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:36.763575 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:37.553285 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:37.553239 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:38.275891 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:38.275847 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:38.375424 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:38.375393 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:46.435183 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:46.435154 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" Sep 06 10:41:46.435708 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:46.435223 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" Sep 06 10:41:46.851543 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:46.851455 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:53.241494 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:53.241462 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:53.801019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:53.800985 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:53.842840 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:53.842806 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc"] Sep 06 10:41:53.843093 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:53.843064 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" podUID="99945164-d23b-4ade-a6d2-3707c354c057" containerName="payload-pre-processing" containerID="cri-o://2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688" gracePeriod=30 Sep 06 10:41:53.900990 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:53.900963 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:54.006814 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.006763 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:54.098001 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.097979 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" Sep 06 10:41:54.101945 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.101897 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:54.179934 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.179904 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:54.222153 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.222111 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6kmn\" (UniqueName: \"kubernetes.io/projected/99945164-d23b-4ade-a6d2-3707c354c057-kube-api-access-t6kmn\") pod \"99945164-d23b-4ade-a6d2-3707c354c057\" (UID: \"99945164-d23b-4ade-a6d2-3707c354c057\") " Sep 06 10:41:54.222318 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.222193 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99945164-d23b-4ade-a6d2-3707c354c057-plugins-config-volume\") pod \"99945164-d23b-4ade-a6d2-3707c354c057\" (UID: \"99945164-d23b-4ade-a6d2-3707c354c057\") " Sep 06 10:41:54.222573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.222550 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99945164-d23b-4ade-a6d2-3707c354c057-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "99945164-d23b-4ade-a6d2-3707c354c057" (UID: "99945164-d23b-4ade-a6d2-3707c354c057"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:41:54.224345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.224325 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99945164-d23b-4ade-a6d2-3707c354c057-kube-api-access-t6kmn" (OuterVolumeSpecName: "kube-api-access-t6kmn") pod "99945164-d23b-4ade-a6d2-3707c354c057" (UID: "99945164-d23b-4ade-a6d2-3707c354c057"). InnerVolumeSpecName "kube-api-access-t6kmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:41:54.322967 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.322932 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6kmn\" (UniqueName: \"kubernetes.io/projected/99945164-d23b-4ade-a6d2-3707c354c057-kube-api-access-t6kmn\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:41:54.322967 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.322960 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99945164-d23b-4ade-a6d2-3707c354c057-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:41:54.541204 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.541164 2574 generic.go:356] "Generic (PLEG): container finished" podID="99945164-d23b-4ade-a6d2-3707c354c057" containerID="2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688" exitCode=0 Sep 06 10:41:54.541378 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.541218 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" Sep 06 10:41:54.541378 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.541250 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" event={"ID":"99945164-d23b-4ade-a6d2-3707c354c057","Type":"ContainerDied","Data":"2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688"} Sep 06 10:41:54.541378 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.541306 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc" event={"ID":"99945164-d23b-4ade-a6d2-3707c354c057","Type":"ContainerDied","Data":"07a01072da5cb3de1eb35e29ba1e3c89ad93133715fef5e6bafc5012b662e518"} Sep 06 10:41:54.541378 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.541329 2574 scope.go:117] "RemoveContainer" containerID="2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688" Sep 06 10:41:54.550475 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.550458 2574 scope.go:117] "RemoveContainer" containerID="2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688" Sep 06 10:41:54.550732 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:41:54.550717 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688\": container with ID starting with 2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688 not found: ID does not exist" containerID="2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688" Sep 06 10:41:54.550778 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.550740 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688"} err="failed to get container status \"2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688\": rpc error: code = NotFound desc = could not find container \"2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688\": container with ID starting with 2e3e545ab8a00a9dcc0120e076c0c28ba8fed0f41d5904b598a8d32062379688 not found: ID does not exist" Sep 06 10:41:54.558530 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.558511 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:54.570569 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.570540 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc"] Sep 06 10:41:54.578581 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.578558 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e0690faf-6887984d7d-bjtzc"] Sep 06 10:41:54.655458 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.655422 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:54.754419 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.754386 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:54.855655 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.855566 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:54.927347 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:54.927315 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99945164-d23b-4ade-a6d2-3707c354c057" path="/var/lib/kubelet/pods/99945164-d23b-4ade-a6d2-3707c354c057/volumes" Sep 06 10:41:55.157822 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:55.157742 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:55.746065 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:55.746033 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:57.370859 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:57.370825 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:58.167016 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:58.166981 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:58.267923 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:58.267889 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:58.343547 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:58.343515 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:58.442787 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:58.442696 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:58.935872 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:58.935841 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:59.029362 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.029330 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:41:59.044576 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.044544 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5"] Sep 06 10:41:59.044948 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.044936 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99945164-d23b-4ade-a6d2-3707c354c057" containerName="payload-pre-processing" Sep 06 10:41:59.044999 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.044949 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="99945164-d23b-4ade-a6d2-3707c354c057" containerName="payload-pre-processing" Sep 06 10:41:59.045038 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.045024 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="99945164-d23b-4ade-a6d2-3707c354c057" containerName="payload-pre-processing" Sep 06 10:41:59.049661 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.049643 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" Sep 06 10:41:59.052068 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.052046 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-2dae43f5-dockercfg-tdjst\"" Sep 06 10:41:59.052203 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.052047 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-2dae43f5\"" Sep 06 10:41:59.057837 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.057815 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5"] Sep 06 10:41:59.173584 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.173405 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddvbv\" (UniqueName: \"kubernetes.io/projected/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5-kube-api-access-ddvbv\") pod \"payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5\" (UID: \"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" Sep 06 10:41:59.173584 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.173520 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5\" (UID: \"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" Sep 06 10:41:59.274969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.274888 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddvbv\" (UniqueName: \"kubernetes.io/projected/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5-kube-api-access-ddvbv\") pod \"payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5\" (UID: \"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" Sep 06 10:41:59.274969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.274935 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5\" (UID: \"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" Sep 06 10:41:59.275662 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.275642 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5\" (UID: \"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" Sep 06 10:41:59.285752 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.285723 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddvbv\" (UniqueName: \"kubernetes.io/projected/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5-kube-api-access-ddvbv\") pod \"payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5\" (UID: \"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" Sep 06 10:41:59.369891 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.369844 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" Sep 06 10:41:59.641283 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:41:59.641250 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5"] Sep 06 10:41:59.642621 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:41:59.642596 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fda9ae_f32d_4fd7_af83_0f7740ebe7e5.slice/crio-0f91904d6166cf37145b701dcfa751aa39b41dae9e590a7b58febf6856d6d2a6 WatchSource:0}: Error finding container 0f91904d6166cf37145b701dcfa751aa39b41dae9e590a7b58febf6856d6d2a6: Status 404 returned error can't find the container with id 0f91904d6166cf37145b701dcfa751aa39b41dae9e590a7b58febf6856d6d2a6 Sep 06 10:42:00.578903 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:00.578871 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" event={"ID":"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5","Type":"ContainerStarted","Data":"53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42"} Sep 06 10:42:00.579083 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:00.578920 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" event={"ID":"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5","Type":"ContainerStarted","Data":"0f91904d6166cf37145b701dcfa751aa39b41dae9e590a7b58febf6856d6d2a6"} Sep 06 10:42:00.579083 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:00.579062 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" Sep 06 10:42:00.595669 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:00.595623 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" podStartSLOduration=1.5956098239999998 podStartE2EDuration="1.595609824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:41:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:42:00.594062315 +0000 UTC m=+714.176551946" watchObservedRunningTime="2026-09-06 10:42:00.595609824 +0000 UTC m=+714.178099455" Sep 06 10:42:04.995089 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:04.995053 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:11.587054 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:11.587022 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" Sep 06 10:42:12.279972 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:12.279940 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:12.372801 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:12.372766 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:12.551419 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:12.551342 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:18.047545 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:18.047508 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:18.150465 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:18.150432 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:18.249118 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:18.249088 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:18.406169 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:18.406061 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:24.572922 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:24.572889 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:24.669722 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:24.669682 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:29.078654 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.078613 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:29.114396 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.114358 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5"] Sep 06 10:42:29.114623 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.114593 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" podUID="c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5" containerName="payload-pre-processing" containerID="cri-o://53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42" gracePeriod=30 Sep 06 10:42:29.181792 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.181756 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:29.278774 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.278742 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:29.339510 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.339412 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:29.388171 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.387844 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" Sep 06 10:42:29.437988 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.437942 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:29.439986 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.439965 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddvbv\" (UniqueName: \"kubernetes.io/projected/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5-kube-api-access-ddvbv\") pod \"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5\" (UID: \"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5\") " Sep 06 10:42:29.440108 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.440028 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5-plugins-config-volume\") pod \"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5\" (UID: \"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5\") " Sep 06 10:42:29.440399 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.440376 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5" (UID: "c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:42:29.441995 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.441969 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5-kube-api-access-ddvbv" (OuterVolumeSpecName: "kube-api-access-ddvbv") pod "c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5" (UID: "c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5"). InnerVolumeSpecName "kube-api-access-ddvbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:42:29.541276 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.541244 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddvbv\" (UniqueName: \"kubernetes.io/projected/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5-kube-api-access-ddvbv\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:42:29.541276 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.541271 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:42:29.742749 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.742715 2574 generic.go:356] "Generic (PLEG): container finished" podID="c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5" containerID="53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42" exitCode=0 Sep 06 10:42:29.742951 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.742758 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" Sep 06 10:42:29.742951 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.742803 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" event={"ID":"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5","Type":"ContainerDied","Data":"53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42"} Sep 06 10:42:29.742951 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.742858 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5" event={"ID":"c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5","Type":"ContainerDied","Data":"0f91904d6166cf37145b701dcfa751aa39b41dae9e590a7b58febf6856d6d2a6"} Sep 06 10:42:29.742951 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.742886 2574 scope.go:117] "RemoveContainer" containerID="53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42" Sep 06 10:42:29.752479 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.752464 2574 scope.go:117] "RemoveContainer" containerID="53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42" Sep 06 10:42:29.752734 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:42:29.752716 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42\": container with ID starting with 53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42 not found: ID does not exist" containerID="53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42" Sep 06 10:42:29.752787 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.752743 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42"} err="failed to get container status \"53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42\": rpc error: code = NotFound desc = could not find container \"53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42\": container with ID starting with 53350cc1362a918d20d5197b3b69b56aba514b658e92cdf926486b17304f8f42 not found: ID does not exist" Sep 06 10:42:29.765539 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.765516 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5"] Sep 06 10:42:29.773185 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.773165 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2dae43f5-66d774666-d8gk5"] Sep 06 10:42:29.850907 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:29.850874 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:30.153372 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:30.153269 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:30.406056 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:30.405971 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:30.927267 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:30.927233 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5" path="/var/lib/kubelet/pods/c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5/volumes" Sep 06 10:42:32.241978 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:32.241944 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:32.334144 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:32.334093 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:33.461492 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:33.461456 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:33.515289 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:33.515259 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:33.933917 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:33.933885 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:34.034061 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.034031 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:34.035461 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.035434 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46"] Sep 06 10:42:34.035908 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.035890 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5" containerName="payload-pre-processing" Sep 06 10:42:34.035997 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.035911 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5" containerName="payload-pre-processing" Sep 06 10:42:34.036060 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.035999 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6fda9ae-f32d-4fd7-af83-0f7740ebe7e5" containerName="payload-pre-processing" Sep 06 10:42:34.038798 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.038775 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" Sep 06 10:42:34.041682 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.041661 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-2b45b43b-dockercfg-gbwc8\"" Sep 06 10:42:34.042011 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.041996 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-2b45b43b\"" Sep 06 10:42:34.059013 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.058990 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46"] Sep 06 10:42:34.081160 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.081114 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsz7w\" (UniqueName: \"kubernetes.io/projected/f50bd585-eed6-483c-ae0f-824960a51d7b-kube-api-access-wsz7w\") pod \"payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46\" (UID: \"f50bd585-eed6-483c-ae0f-824960a51d7b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" Sep 06 10:42:34.081342 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.081190 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f50bd585-eed6-483c-ae0f-824960a51d7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46\" (UID: \"f50bd585-eed6-483c-ae0f-824960a51d7b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" Sep 06 10:42:34.161029 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.160987 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:42:34.182187 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.182156 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f50bd585-eed6-483c-ae0f-824960a51d7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46\" (UID: \"f50bd585-eed6-483c-ae0f-824960a51d7b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" Sep 06 10:42:34.182407 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.182271 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsz7w\" (UniqueName: \"kubernetes.io/projected/f50bd585-eed6-483c-ae0f-824960a51d7b-kube-api-access-wsz7w\") pod \"payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46\" (UID: \"f50bd585-eed6-483c-ae0f-824960a51d7b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" Sep 06 10:42:34.182902 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.182877 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f50bd585-eed6-483c-ae0f-824960a51d7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46\" (UID: \"f50bd585-eed6-483c-ae0f-824960a51d7b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" Sep 06 10:42:34.198304 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.198238 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsz7w\" (UniqueName: \"kubernetes.io/projected/f50bd585-eed6-483c-ae0f-824960a51d7b-kube-api-access-wsz7w\") pod \"payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46\" (UID: \"f50bd585-eed6-483c-ae0f-824960a51d7b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" Sep 06 10:42:34.348956 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.348919 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" Sep 06 10:42:34.618428 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.618403 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46"] Sep 06 10:42:34.619825 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:42:34.619797 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf50bd585_eed6_483c_ae0f_824960a51d7b.slice/crio-dd80c83b4372cb147d342c168c0e8ed5e9cc66883a2ac5f85f49ea75b4c2537a WatchSource:0}: Error finding container dd80c83b4372cb147d342c168c0e8ed5e9cc66883a2ac5f85f49ea75b4c2537a: Status 404 returned error can't find the container with id dd80c83b4372cb147d342c168c0e8ed5e9cc66883a2ac5f85f49ea75b4c2537a Sep 06 10:42:34.777049 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.777013 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" event={"ID":"f50bd585-eed6-483c-ae0f-824960a51d7b","Type":"ContainerStarted","Data":"21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c"} Sep 06 10:42:34.777237 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.777069 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" event={"ID":"f50bd585-eed6-483c-ae0f-824960a51d7b","Type":"ContainerStarted","Data":"dd80c83b4372cb147d342c168c0e8ed5e9cc66883a2ac5f85f49ea75b4c2537a"} Sep 06 10:42:34.777237 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.777096 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" Sep 06 10:42:34.793024 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:34.792979 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" podStartSLOduration=0.792964553 podStartE2EDuration="792.964553ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:42:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:42:34.790881967 +0000 UTC m=+748.373371597" watchObservedRunningTime="2026-09-06 10:42:34.792964553 +0000 UTC m=+748.375454183" Sep 06 10:42:45.784966 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:45.784932 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" Sep 06 10:42:55.151611 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:42:55.151581 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:03.326299 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.326267 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r"] Sep 06 10:43:03.329653 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.329633 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.331727 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.331705 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-511895d8-kserve-self-signed-certs\"" Sep 06 10:43:03.341828 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.341803 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r"] Sep 06 10:43:03.363242 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.363217 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:03.453733 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.453697 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/087cd4cc-7ae0-450e-9c23-8b152d4176cf-tls-certs\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.453979 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.453961 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-tmp-dir\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.454179 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.454160 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-dshm\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.454285 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.454271 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-home\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.454388 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.454376 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-model-cache\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.454525 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.454512 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rcjl\" (UniqueName: \"kubernetes.io/projected/087cd4cc-7ae0-450e-9c23-8b152d4176cf-kube-api-access-4rcjl\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.555839 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.555806 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-dshm\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.555839 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.555841 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-home\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.556019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.555863 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-model-cache\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.556019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.555906 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rcjl\" (UniqueName: \"kubernetes.io/projected/087cd4cc-7ae0-450e-9c23-8b152d4176cf-kube-api-access-4rcjl\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.556019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.555949 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/087cd4cc-7ae0-450e-9c23-8b152d4176cf-tls-certs\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.556194 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.556070 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-tmp-dir\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.556339 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.556312 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-home\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.556414 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.556350 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-model-cache\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.556414 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.556392 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-tmp-dir\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.558099 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.558071 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-dshm\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.558424 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.558407 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/087cd4cc-7ae0-450e-9c23-8b152d4176cf-tls-certs\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.562760 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.562736 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rcjl\" (UniqueName: \"kubernetes.io/projected/087cd4cc-7ae0-450e-9c23-8b152d4176cf-kube-api-access-4rcjl\") pod \"e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.641165 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.641081 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:03.930049 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.930019 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r"] Sep 06 10:43:03.930977 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:43:03.930947 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod087cd4cc_7ae0_450e_9c23_8b152d4176cf.slice/crio-c2ed009a279084483d5f39b211682433578afe55ae29dbba07bda2eee531f0fa WatchSource:0}: Error finding container c2ed009a279084483d5f39b211682433578afe55ae29dbba07bda2eee531f0fa: Status 404 returned error can't find the container with id c2ed009a279084483d5f39b211682433578afe55ae29dbba07bda2eee531f0fa Sep 06 10:43:03.944191 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:03.944168 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" event={"ID":"087cd4cc-7ae0-450e-9c23-8b152d4176cf","Type":"ContainerStarted","Data":"c2ed009a279084483d5f39b211682433578afe55ae29dbba07bda2eee531f0fa"} Sep 06 10:43:04.206039 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.205954 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:04.261860 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.261818 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46"] Sep 06 10:43:04.262101 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.262047 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" podUID="f50bd585-eed6-483c-ae0f-824960a51d7b" containerName="payload-pre-processing" containerID="cri-o://21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c" gracePeriod=30 Sep 06 10:43:04.280909 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.280882 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:04.342767 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.342733 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:04.500210 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.500178 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:04.501975 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.501957 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" Sep 06 10:43:04.559414 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.559380 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:04.668515 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.668482 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsz7w\" (UniqueName: \"kubernetes.io/projected/f50bd585-eed6-483c-ae0f-824960a51d7b-kube-api-access-wsz7w\") pod \"f50bd585-eed6-483c-ae0f-824960a51d7b\" (UID: \"f50bd585-eed6-483c-ae0f-824960a51d7b\") " Sep 06 10:43:04.668714 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.668565 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f50bd585-eed6-483c-ae0f-824960a51d7b-plugins-config-volume\") pod \"f50bd585-eed6-483c-ae0f-824960a51d7b\" (UID: \"f50bd585-eed6-483c-ae0f-824960a51d7b\") " Sep 06 10:43:04.668932 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.668909 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f50bd585-eed6-483c-ae0f-824960a51d7b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f50bd585-eed6-483c-ae0f-824960a51d7b" (UID: "f50bd585-eed6-483c-ae0f-824960a51d7b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:43:04.670759 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.670734 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f50bd585-eed6-483c-ae0f-824960a51d7b-kube-api-access-wsz7w" (OuterVolumeSpecName: "kube-api-access-wsz7w") pod "f50bd585-eed6-483c-ae0f-824960a51d7b" (UID: "f50bd585-eed6-483c-ae0f-824960a51d7b"). InnerVolumeSpecName "kube-api-access-wsz7w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:43:04.769735 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.769650 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsz7w\" (UniqueName: \"kubernetes.io/projected/f50bd585-eed6-483c-ae0f-824960a51d7b-kube-api-access-wsz7w\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:04.769735 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.769680 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f50bd585-eed6-483c-ae0f-824960a51d7b-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:04.951814 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.951782 2574 generic.go:356] "Generic (PLEG): container finished" podID="f50bd585-eed6-483c-ae0f-824960a51d7b" containerID="21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c" exitCode=0 Sep 06 10:43:04.951999 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.951835 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" Sep 06 10:43:04.951999 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.951858 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" event={"ID":"f50bd585-eed6-483c-ae0f-824960a51d7b","Type":"ContainerDied","Data":"21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c"} Sep 06 10:43:04.951999 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.951898 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46" event={"ID":"f50bd585-eed6-483c-ae0f-824960a51d7b","Type":"ContainerDied","Data":"dd80c83b4372cb147d342c168c0e8ed5e9cc66883a2ac5f85f49ea75b4c2537a"} Sep 06 10:43:04.951999 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.951924 2574 scope.go:117] "RemoveContainer" containerID="21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c" Sep 06 10:43:04.953731 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.953708 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" event={"ID":"087cd4cc-7ae0-450e-9c23-8b152d4176cf","Type":"ContainerStarted","Data":"8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b"} Sep 06 10:43:04.961228 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.961212 2574 scope.go:117] "RemoveContainer" containerID="21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c" Sep 06 10:43:04.961507 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:43:04.961483 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c\": container with ID starting with 21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c not found: ID does not exist" containerID="21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c" Sep 06 10:43:04.961571 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.961514 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c"} err="failed to get container status \"21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c\": rpc error: code = NotFound desc = could not find container \"21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c\": container with ID starting with 21bb1c4a4d29a5a6ad9ff0fd8c0537e6581a16c92b88d4bd11deb762b061587c not found: ID does not exist" Sep 06 10:43:04.969994 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.969956 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" podStartSLOduration=1.9699446809999999 podStartE2EDuration="1.969944681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:43:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:43:04.96704881 +0000 UTC m=+778.549538443" watchObservedRunningTime="2026-09-06 10:43:04.969944681 +0000 UTC m=+778.552434311" Sep 06 10:43:04.977957 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.977939 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46"] Sep 06 10:43:04.980152 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:04.980114 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2b45b43b-79b4cdf4-8dk46"] Sep 06 10:43:05.627808 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:05.627777 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:06.061462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:06.061432 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:06.321310 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:06.321228 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:06.559627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:06.559594 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:06.926435 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:06.926403 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f50bd585-eed6-483c-ae0f-824960a51d7b" path="/var/lib/kubelet/pods/f50bd585-eed6-483c-ae0f-824960a51d7b/volumes" Sep 06 10:43:08.540001 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:08.539962 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:08.639598 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:08.639563 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:09.690054 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:09.690015 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:09.791447 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:09.791413 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:09.965412 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:09.965331 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:10.280801 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.280722 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:10.371523 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.371459 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:10.388456 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.388421 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw"] Sep 06 10:43:10.388909 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.388891 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f50bd585-eed6-483c-ae0f-824960a51d7b" containerName="payload-pre-processing" Sep 06 10:43:10.388989 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.388911 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50bd585-eed6-483c-ae0f-824960a51d7b" containerName="payload-pre-processing" Sep 06 10:43:10.389053 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.388995 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f50bd585-eed6-483c-ae0f-824960a51d7b" containerName="payload-pre-processing" Sep 06 10:43:10.393660 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.393639 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" Sep 06 10:43:10.398643 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.398615 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-02ec029f\"" Sep 06 10:43:10.399051 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.399030 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-02ec029f-dockercfg-lf6d4\"" Sep 06 10:43:10.405339 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.405316 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw"] Sep 06 10:43:10.421859 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.421836 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw\" (UID: \"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" Sep 06 10:43:10.421976 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.421920 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8zzw\" (UniqueName: \"kubernetes.io/projected/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8-kube-api-access-q8zzw\") pod \"payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw\" (UID: \"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" Sep 06 10:43:10.523026 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.522994 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw\" (UID: \"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" Sep 06 10:43:10.523218 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.523082 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8zzw\" (UniqueName: \"kubernetes.io/projected/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8-kube-api-access-q8zzw\") pod \"payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw\" (UID: \"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" Sep 06 10:43:10.523706 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.523688 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw\" (UID: \"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" Sep 06 10:43:10.530635 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.530607 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8zzw\" (UniqueName: \"kubernetes.io/projected/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8-kube-api-access-q8zzw\") pod \"payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw\" (UID: \"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" Sep 06 10:43:10.705071 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.705034 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" Sep 06 10:43:10.977467 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.977435 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw"] Sep 06 10:43:10.978560 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:43:10.978536 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad7c12d_9f34_4c3b_9478_dceebd7c1fd8.slice/crio-cc1e3f795850c728d06e120acfdf72efd413bae8883dca097ef135727303d50c WatchSource:0}: Error finding container cc1e3f795850c728d06e120acfdf72efd413bae8883dca097ef135727303d50c: Status 404 returned error can't find the container with id cc1e3f795850c728d06e120acfdf72efd413bae8883dca097ef135727303d50c Sep 06 10:43:10.989744 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:10.989721 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" event={"ID":"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8","Type":"ContainerStarted","Data":"cc1e3f795850c728d06e120acfdf72efd413bae8883dca097ef135727303d50c"} Sep 06 10:43:11.997567 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:11.997526 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" event={"ID":"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8","Type":"ContainerStarted","Data":"06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26"} Sep 06 10:43:11.998014 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:11.997585 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" Sep 06 10:43:12.013413 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:12.013360 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" podStartSLOduration=2.013345509 podStartE2EDuration="2.013345509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:43:12.010557895 +0000 UTC m=+785.593047527" watchObservedRunningTime="2026-09-06 10:43:12.013345509 +0000 UTC m=+785.595835163" Sep 06 10:43:13.641282 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:13.641237 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:13.641704 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:13.641298 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:13.654853 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:13.654828 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:14.019433 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:14.019399 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:23.005225 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:23.005190 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" Sep 06 10:43:26.879851 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:26.879814 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:29.028013 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.027977 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g"] Sep 06 10:43:29.032643 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.032621 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.034942 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.034916 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-232aeb\"/\"kube-root-ca.crt\"" Sep 06 10:43:29.035962 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.035939 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-232aeb\"/\"openshift-service-ca.crt\"" Sep 06 10:43:29.036249 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.035945 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-232aeb\"/\"test-model-232aeb-kserve-self-signed-certs\"" Sep 06 10:43:29.036351 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.035990 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-232aeb\"/\"default-dockercfg-m8876\"" Sep 06 10:43:29.039196 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.039176 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g"] Sep 06 10:43:29.054154 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.054108 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:29.096007 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.095977 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-model-cache\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.096156 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.096012 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-tmp-dir\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.096156 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.096094 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-dshm\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.096282 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.096187 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fv4h\" (UniqueName: \"kubernetes.io/projected/ff67ccc1-a644-4940-891d-e2ee689c1c54-kube-api-access-5fv4h\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.096282 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.096245 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff67ccc1-a644-4940-891d-e2ee689c1c54-tls-certs\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.096364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.096302 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-home\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.197462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.197372 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fv4h\" (UniqueName: \"kubernetes.io/projected/ff67ccc1-a644-4940-891d-e2ee689c1c54-kube-api-access-5fv4h\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.198096 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.198068 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff67ccc1-a644-4940-891d-e2ee689c1c54-tls-certs\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.198934 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.198916 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-home\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.199613 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.199415 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-home\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.199613 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.199541 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-model-cache\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.199806 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.199628 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-tmp-dir\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.199806 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.199733 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-dshm\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.199806 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.199792 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-model-cache\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.199967 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.199947 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-tmp-dir\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.203104 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.203081 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-dshm\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.203210 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.203089 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff67ccc1-a644-4940-891d-e2ee689c1c54-tls-certs\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.205654 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.205627 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fv4h\" (UniqueName: \"kubernetes.io/projected/ff67ccc1-a644-4940-891d-e2ee689c1c54-kube-api-access-5fv4h\") pod \"test-model-232aeb-kserve-7bfd5766dc-2sr2g\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.344296 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.344220 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:29.619137 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:29.619074 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g"] Sep 06 10:43:30.099021 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:30.098987 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" event={"ID":"ff67ccc1-a644-4940-891d-e2ee689c1c54","Type":"ContainerStarted","Data":"d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd"} Sep 06 10:43:30.099404 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:30.099029 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" event={"ID":"ff67ccc1-a644-4940-891d-e2ee689c1c54","Type":"ContainerStarted","Data":"6899f05c8ba01f4454dcba6379a065b84c37e541f9510cd407c458f6dcb1cc67"} Sep 06 10:43:30.114399 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:30.114351 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" podStartSLOduration=1.114336266 podStartE2EDuration="1.114336266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:43:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:43:30.113755381 +0000 UTC m=+803.696245013" watchObservedRunningTime="2026-09-06 10:43:30.114336266 +0000 UTC m=+803.696825897" Sep 06 10:43:30.769670 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:30.769628 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:31.167435 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:31.167409 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:32.425485 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:32.425450 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r"] Sep 06 10:43:32.445001 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:32.444971 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:36.676961 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:36.676918 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:36.929931 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:36.929856 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:37.499566 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.499534 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:37.525285 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.525250 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg"] Sep 06 10:43:37.525520 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.525485 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" podUID="97e8df12-9ca3-4b1e-804d-88e9812c9417" containerName="payload-processing" containerID="cri-o://44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b" gracePeriod=30 Sep 06 10:43:37.534658 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.534619 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487"] Sep 06 10:43:37.534894 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.534842 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" podUID="166d191f-bb72-4ff6-b12d-f58873442f1e" containerName="payload-pre-processing" containerID="cri-o://60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4" gracePeriod=30 Sep 06 10:43:37.563957 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.563893 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:37.622958 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.622922 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:37.697041 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.697005 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:37.749822 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.749740 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:37.821445 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.821424 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" Sep 06 10:43:37.824214 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.824195 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" Sep 06 10:43:37.993411 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.993379 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e8df12-9ca3-4b1e-804d-88e9812c9417-plugins-config-volume\") pod \"97e8df12-9ca3-4b1e-804d-88e9812c9417\" (UID: \"97e8df12-9ca3-4b1e-804d-88e9812c9417\") " Sep 06 10:43:37.993576 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.993444 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndsth\" (UniqueName: \"kubernetes.io/projected/97e8df12-9ca3-4b1e-804d-88e9812c9417-kube-api-access-ndsth\") pod \"97e8df12-9ca3-4b1e-804d-88e9812c9417\" (UID: \"97e8df12-9ca3-4b1e-804d-88e9812c9417\") " Sep 06 10:43:37.993576 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.993466 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/166d191f-bb72-4ff6-b12d-f58873442f1e-plugins-config-volume\") pod \"166d191f-bb72-4ff6-b12d-f58873442f1e\" (UID: \"166d191f-bb72-4ff6-b12d-f58873442f1e\") " Sep 06 10:43:37.993576 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.993501 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k956f\" (UniqueName: \"kubernetes.io/projected/166d191f-bb72-4ff6-b12d-f58873442f1e-kube-api-access-k956f\") pod \"166d191f-bb72-4ff6-b12d-f58873442f1e\" (UID: \"166d191f-bb72-4ff6-b12d-f58873442f1e\") " Sep 06 10:43:37.993819 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.993792 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97e8df12-9ca3-4b1e-804d-88e9812c9417-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "97e8df12-9ca3-4b1e-804d-88e9812c9417" (UID: "97e8df12-9ca3-4b1e-804d-88e9812c9417"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:43:37.993888 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.993838 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/166d191f-bb72-4ff6-b12d-f58873442f1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "166d191f-bb72-4ff6-b12d-f58873442f1e" (UID: "166d191f-bb72-4ff6-b12d-f58873442f1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:43:37.995501 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.995479 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97e8df12-9ca3-4b1e-804d-88e9812c9417-kube-api-access-ndsth" (OuterVolumeSpecName: "kube-api-access-ndsth") pod "97e8df12-9ca3-4b1e-804d-88e9812c9417" (UID: "97e8df12-9ca3-4b1e-804d-88e9812c9417"). InnerVolumeSpecName "kube-api-access-ndsth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:43:37.995580 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:37.995541 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/166d191f-bb72-4ff6-b12d-f58873442f1e-kube-api-access-k956f" (OuterVolumeSpecName: "kube-api-access-k956f") pod "166d191f-bb72-4ff6-b12d-f58873442f1e" (UID: "166d191f-bb72-4ff6-b12d-f58873442f1e"). InnerVolumeSpecName "kube-api-access-k956f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:43:38.055962 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.055888 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:38.094837 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.094803 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k956f\" (UniqueName: \"kubernetes.io/projected/166d191f-bb72-4ff6-b12d-f58873442f1e-kube-api-access-k956f\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:38.094837 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.094833 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97e8df12-9ca3-4b1e-804d-88e9812c9417-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:38.095043 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.094846 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndsth\" (UniqueName: \"kubernetes.io/projected/97e8df12-9ca3-4b1e-804d-88e9812c9417-kube-api-access-ndsth\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:38.095043 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.094858 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/166d191f-bb72-4ff6-b12d-f58873442f1e-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:38.143169 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.143137 2574 generic.go:356] "Generic (PLEG): container finished" podID="166d191f-bb72-4ff6-b12d-f58873442f1e" containerID="60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4" exitCode=0 Sep 06 10:43:38.143306 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.143188 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" Sep 06 10:43:38.143306 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.143215 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" event={"ID":"166d191f-bb72-4ff6-b12d-f58873442f1e","Type":"ContainerDied","Data":"60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4"} Sep 06 10:43:38.143306 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.143266 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487" event={"ID":"166d191f-bb72-4ff6-b12d-f58873442f1e","Type":"ContainerDied","Data":"be10f6e2ef40608ec02133f875b039028b5e8328e7dba9d190882e7c257b0b39"} Sep 06 10:43:38.143306 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.143292 2574 scope.go:117] "RemoveContainer" containerID="60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4" Sep 06 10:43:38.144887 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.144854 2574 generic.go:356] "Generic (PLEG): container finished" podID="97e8df12-9ca3-4b1e-804d-88e9812c9417" containerID="44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b" exitCode=0 Sep 06 10:43:38.144994 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.144894 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" Sep 06 10:43:38.144994 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.144904 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" event={"ID":"97e8df12-9ca3-4b1e-804d-88e9812c9417","Type":"ContainerDied","Data":"44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b"} Sep 06 10:43:38.144994 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.144944 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg" event={"ID":"97e8df12-9ca3-4b1e-804d-88e9812c9417","Type":"ContainerDied","Data":"bdada11f65b36fc34930b85893657bf47bf6a4149dee29acdddab4316cf6a913"} Sep 06 10:43:38.154201 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.154177 2574 scope.go:117] "RemoveContainer" containerID="60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4" Sep 06 10:43:38.154501 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:43:38.154482 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4\": container with ID starting with 60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4 not found: ID does not exist" containerID="60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4" Sep 06 10:43:38.154594 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.154510 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4"} err="failed to get container status \"60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4\": rpc error: code = NotFound desc = could not find container \"60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4\": container with ID starting with 60deb9f3888592090bf4cbf7b1c5bac76c75f87ac4854c6f15f86d2a36b18bd4 not found: ID does not exist" Sep 06 10:43:38.154594 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.154532 2574 scope.go:117] "RemoveContainer" containerID="44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b" Sep 06 10:43:38.163656 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.163636 2574 scope.go:117] "RemoveContainer" containerID="44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b" Sep 06 10:43:38.163885 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:43:38.163869 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b\": container with ID starting with 44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b not found: ID does not exist" containerID="44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b" Sep 06 10:43:38.163934 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.163890 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b"} err="failed to get container status \"44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b\": rpc error: code = NotFound desc = could not find container \"44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b\": container with ID starting with 44fe8aa7ec287a05bf353684f6604f5b2f8f5dbebb7a0d5373e64c471be1395b not found: ID does not exist" Sep 06 10:43:38.165231 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.165213 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487"] Sep 06 10:43:38.168489 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.168467 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-55111c-d7b6f9fdb-m5487"] Sep 06 10:43:38.178200 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.178176 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg"] Sep 06 10:43:38.184572 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.184549 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-55111c-bc5896cc-cj2lg"] Sep 06 10:43:38.927402 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.927371 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="166d191f-bb72-4ff6-b12d-f58873442f1e" path="/var/lib/kubelet/pods/166d191f-bb72-4ff6-b12d-f58873442f1e/volumes" Sep 06 10:43:38.927782 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:38.927768 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97e8df12-9ca3-4b1e-804d-88e9812c9417" path="/var/lib/kubelet/pods/97e8df12-9ca3-4b1e-804d-88e9812c9417/volumes" Sep 06 10:43:39.066616 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:39.066585 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:39.344363 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:39.344283 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:39.344363 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:39.344344 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:39.357769 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:39.357746 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:39.959288 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:39.959253 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:40.170315 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.170283 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:43:40.527963 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.527923 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:40.570135 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.568902 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw"] Sep 06 10:43:40.570135 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.569217 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" podUID="1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8" containerName="payload-pre-processing" containerID="cri-o://06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26" gracePeriod=30 Sep 06 10:43:40.592316 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.592286 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:40.653035 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.653002 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:40.717292 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.717257 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:40.829599 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.829567 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" Sep 06 10:43:40.856313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.856272 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:40.921835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.921807 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8zzw\" (UniqueName: \"kubernetes.io/projected/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8-kube-api-access-q8zzw\") pod \"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8\" (UID: \"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8\") " Sep 06 10:43:40.922008 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.921861 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8-plugins-config-volume\") pod \"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8\" (UID: \"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8\") " Sep 06 10:43:40.922309 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.922280 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8" (UID: "1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:43:40.924028 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:40.923997 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8-kube-api-access-q8zzw" (OuterVolumeSpecName: "kube-api-access-q8zzw") pod "1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8" (UID: "1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8"). InnerVolumeSpecName "kube-api-access-q8zzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:43:41.023025 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.022990 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:41.023025 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.023019 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8zzw\" (UniqueName: \"kubernetes.io/projected/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8-kube-api-access-q8zzw\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:41.166196 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.166167 2574 generic.go:356] "Generic (PLEG): container finished" podID="1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8" containerID="06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26" exitCode=0 Sep 06 10:43:41.166350 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.166212 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" Sep 06 10:43:41.166350 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.166212 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" event={"ID":"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8","Type":"ContainerDied","Data":"06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26"} Sep 06 10:43:41.166350 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.166324 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw" event={"ID":"1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8","Type":"ContainerDied","Data":"cc1e3f795850c728d06e120acfdf72efd413bae8883dca097ef135727303d50c"} Sep 06 10:43:41.166350 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.166346 2574 scope.go:117] "RemoveContainer" containerID="06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26" Sep 06 10:43:41.175549 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.175530 2574 scope.go:117] "RemoveContainer" containerID="06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26" Sep 06 10:43:41.175800 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:43:41.175780 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26\": container with ID starting with 06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26 not found: ID does not exist" containerID="06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26" Sep 06 10:43:41.175862 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.175806 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26"} err="failed to get container status \"06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26\": rpc error: code = NotFound desc = could not find container \"06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26\": container with ID starting with 06c4d37f42d66e8d9bdf5241cdfb0d38c4ae0b3e6cdf42ab8b41cbc374154a26 not found: ID does not exist" Sep 06 10:43:41.182372 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.182351 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw"] Sep 06 10:43:41.186023 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.185992 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-02ec029f-6944458bcd-8s6fw"] Sep 06 10:43:41.484409 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.484286 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:41.583803 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.583751 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:41.684098 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:41.684058 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:42.928438 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:42.928400 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8" path="/var/lib/kubelet/pods/1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8/volumes" Sep 06 10:43:42.979772 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:42.979740 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:43.080641 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:43.080612 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:43.402801 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:43.402693 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:47.446891 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.446832 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" podUID="087cd4cc-7ae0-450e-9c23-8b152d4176cf" containerName="main" containerID="cri-o://8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b" gracePeriod=45 Sep 06 10:43:47.698783 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.698728 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:47.788701 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.788666 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/087cd4cc-7ae0-450e-9c23-8b152d4176cf-tls-certs\") pod \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " Sep 06 10:43:47.788867 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.788712 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-model-cache\") pod \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " Sep 06 10:43:47.788867 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.788742 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-dshm\") pod \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " Sep 06 10:43:47.788867 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.788776 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-tmp-dir\") pod \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " Sep 06 10:43:47.788867 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.788806 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rcjl\" (UniqueName: \"kubernetes.io/projected/087cd4cc-7ae0-450e-9c23-8b152d4176cf-kube-api-access-4rcjl\") pod \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " Sep 06 10:43:47.788867 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.788844 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-home\") pod \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\" (UID: \"087cd4cc-7ae0-450e-9c23-8b152d4176cf\") " Sep 06 10:43:47.789166 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.789051 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-model-cache" (OuterVolumeSpecName: "model-cache") pod "087cd4cc-7ae0-450e-9c23-8b152d4176cf" (UID: "087cd4cc-7ae0-450e-9c23-8b152d4176cf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:43:47.789353 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.789299 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-home" (OuterVolumeSpecName: "home") pod "087cd4cc-7ae0-450e-9c23-8b152d4176cf" (UID: "087cd4cc-7ae0-450e-9c23-8b152d4176cf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:43:47.789353 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.789314 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "087cd4cc-7ae0-450e-9c23-8b152d4176cf" (UID: "087cd4cc-7ae0-450e-9c23-8b152d4176cf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:43:47.789353 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.789342 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-model-cache\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:47.791457 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.791427 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-dshm" (OuterVolumeSpecName: "dshm") pod "087cd4cc-7ae0-450e-9c23-8b152d4176cf" (UID: "087cd4cc-7ae0-450e-9c23-8b152d4176cf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:43:47.791568 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.791425 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/087cd4cc-7ae0-450e-9c23-8b152d4176cf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "087cd4cc-7ae0-450e-9c23-8b152d4176cf" (UID: "087cd4cc-7ae0-450e-9c23-8b152d4176cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:43:47.791666 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.791645 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/087cd4cc-7ae0-450e-9c23-8b152d4176cf-kube-api-access-4rcjl" (OuterVolumeSpecName: "kube-api-access-4rcjl") pod "087cd4cc-7ae0-450e-9c23-8b152d4176cf" (UID: "087cd4cc-7ae0-450e-9c23-8b152d4176cf"). InnerVolumeSpecName "kube-api-access-4rcjl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:43:47.890560 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.890528 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/087cd4cc-7ae0-450e-9c23-8b152d4176cf-tls-certs\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:47.890560 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.890554 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-dshm\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:47.890560 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.890562 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-tmp-dir\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:47.890749 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.890570 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rcjl\" (UniqueName: \"kubernetes.io/projected/087cd4cc-7ae0-450e-9c23-8b152d4176cf-kube-api-access-4rcjl\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:47.890749 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:47.890580 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/087cd4cc-7ae0-450e-9c23-8b152d4176cf-home\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:43:48.214366 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:48.214331 2574 generic.go:356] "Generic (PLEG): container finished" podID="087cd4cc-7ae0-450e-9c23-8b152d4176cf" containerID="8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b" exitCode=0 Sep 06 10:43:48.214531 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:48.214376 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" Sep 06 10:43:48.214531 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:48.214397 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" event={"ID":"087cd4cc-7ae0-450e-9c23-8b152d4176cf","Type":"ContainerDied","Data":"8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b"} Sep 06 10:43:48.214531 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:48.214438 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r" event={"ID":"087cd4cc-7ae0-450e-9c23-8b152d4176cf","Type":"ContainerDied","Data":"c2ed009a279084483d5f39b211682433578afe55ae29dbba07bda2eee531f0fa"} Sep 06 10:43:48.214531 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:48.214463 2574 scope.go:117] "RemoveContainer" containerID="8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b" Sep 06 10:43:48.225999 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:48.225980 2574 scope.go:117] "RemoveContainer" containerID="8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b" Sep 06 10:43:48.226306 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:43:48.226288 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b\": container with ID starting with 8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b not found: ID does not exist" containerID="8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b" Sep 06 10:43:48.226347 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:48.226315 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b"} err="failed to get container status \"8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b\": rpc error: code = NotFound desc = could not find container \"8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b\": container with ID starting with 8c29c817995a0707eaa23dcb906e533220de2f4742409d606fe968ea5956e57b not found: ID does not exist" Sep 06 10:43:48.236735 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:48.236712 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r"] Sep 06 10:43:48.240699 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:48.240677 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-511895d8-kserve-6987b6c7b-lt85r"] Sep 06 10:43:48.926439 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:48.926411 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="087cd4cc-7ae0-450e-9c23-8b152d4176cf" path="/var/lib/kubelet/pods/087cd4cc-7ae0-450e-9c23-8b152d4176cf/volumes" Sep 06 10:43:49.446651 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:49.446624 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:49.626573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:49.626540 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:49.726565 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:49.726487 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:49.826722 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:49.826687 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:49.929571 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:49.929540 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:50.100116 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:50.100031 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:50.299747 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:50.299712 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:50.432202 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:50.432169 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:51.042538 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:51.042509 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:53.294596 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:53.294558 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:53.392283 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:53.392223 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:54.567321 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:54.567286 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:54.624271 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:54.624236 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:54.724398 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:54.724359 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:55.013982 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.013953 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:55.114066 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.114032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:43:55.143696 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.143667 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l"] Sep 06 10:43:55.144072 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.144057 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97e8df12-9ca3-4b1e-804d-88e9812c9417" containerName="payload-processing" Sep 06 10:43:55.144135 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.144076 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="97e8df12-9ca3-4b1e-804d-88e9812c9417" containerName="payload-processing" Sep 06 10:43:55.144135 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.144087 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="087cd4cc-7ae0-450e-9c23-8b152d4176cf" containerName="main" Sep 06 10:43:55.144135 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.144097 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="087cd4cc-7ae0-450e-9c23-8b152d4176cf" containerName="main" Sep 06 10:43:55.144135 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.144114 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8" containerName="payload-pre-processing" Sep 06 10:43:55.144303 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.144150 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8" containerName="payload-pre-processing" Sep 06 10:43:55.144303 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.144160 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="166d191f-bb72-4ff6-b12d-f58873442f1e" containerName="payload-pre-processing" Sep 06 10:43:55.144303 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.144165 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="166d191f-bb72-4ff6-b12d-f58873442f1e" containerName="payload-pre-processing" Sep 06 10:43:55.144303 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.144253 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="087cd4cc-7ae0-450e-9c23-8b152d4176cf" containerName="main" Sep 06 10:43:55.144303 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.144267 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad7c12d-9f34-4c3b-9478-dceebd7c1fd8" containerName="payload-pre-processing" Sep 06 10:43:55.144303 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.144278 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="166d191f-bb72-4ff6-b12d-f58873442f1e" containerName="payload-pre-processing" Sep 06 10:43:55.144303 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.144286 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="97e8df12-9ca3-4b1e-804d-88e9812c9417" containerName="payload-processing" Sep 06 10:43:55.148702 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.148680 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" Sep 06 10:43:55.150883 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.150862 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-ccf833\"" Sep 06 10:43:55.150991 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.150904 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-ccf833-dockercfg-s5zlc\"" Sep 06 10:43:55.158074 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.158053 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l"] Sep 06 10:43:55.260115 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.260087 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm5fs\" (UniqueName: \"kubernetes.io/projected/63800b4f-2b28-4a38-aee2-e70ca0c07161-kube-api-access-hm5fs\") pod \"payload-processing-e2e-explicit-ccf833-745684f765-gm48l\" (UID: \"63800b4f-2b28-4a38-aee2-e70ca0c07161\") " pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" Sep 06 10:43:55.260291 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.260157 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63800b4f-2b28-4a38-aee2-e70ca0c07161-plugins-config-volume\") pod \"payload-processing-e2e-explicit-ccf833-745684f765-gm48l\" (UID: \"63800b4f-2b28-4a38-aee2-e70ca0c07161\") " pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" Sep 06 10:43:55.360848 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.360765 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63800b4f-2b28-4a38-aee2-e70ca0c07161-plugins-config-volume\") pod \"payload-processing-e2e-explicit-ccf833-745684f765-gm48l\" (UID: \"63800b4f-2b28-4a38-aee2-e70ca0c07161\") " pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" Sep 06 10:43:55.360848 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.360841 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm5fs\" (UniqueName: \"kubernetes.io/projected/63800b4f-2b28-4a38-aee2-e70ca0c07161-kube-api-access-hm5fs\") pod \"payload-processing-e2e-explicit-ccf833-745684f765-gm48l\" (UID: \"63800b4f-2b28-4a38-aee2-e70ca0c07161\") " pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" Sep 06 10:43:55.361497 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.361476 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63800b4f-2b28-4a38-aee2-e70ca0c07161-plugins-config-volume\") pod \"payload-processing-e2e-explicit-ccf833-745684f765-gm48l\" (UID: \"63800b4f-2b28-4a38-aee2-e70ca0c07161\") " pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" Sep 06 10:43:55.368297 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.368274 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm5fs\" (UniqueName: \"kubernetes.io/projected/63800b4f-2b28-4a38-aee2-e70ca0c07161-kube-api-access-hm5fs\") pod \"payload-processing-e2e-explicit-ccf833-745684f765-gm48l\" (UID: \"63800b4f-2b28-4a38-aee2-e70ca0c07161\") " pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" Sep 06 10:43:55.459863 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.459831 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" Sep 06 10:43:55.743748 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:55.743720 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l"] Sep 06 10:43:55.745027 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:43:55.744999 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63800b4f_2b28_4a38_aee2_e70ca0c07161.slice/crio-239f961ede3edf3125f7bcf388cd2bccf8ce0c402c6c605bdfcfa97580be2959 WatchSource:0}: Error finding container 239f961ede3edf3125f7bcf388cd2bccf8ce0c402c6c605bdfcfa97580be2959: Status 404 returned error can't find the container with id 239f961ede3edf3125f7bcf388cd2bccf8ce0c402c6c605bdfcfa97580be2959 Sep 06 10:43:56.269315 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:56.269273 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" event={"ID":"63800b4f-2b28-4a38-aee2-e70ca0c07161","Type":"ContainerStarted","Data":"4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65"} Sep 06 10:43:56.269479 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:56.269366 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" event={"ID":"63800b4f-2b28-4a38-aee2-e70ca0c07161","Type":"ContainerStarted","Data":"239f961ede3edf3125f7bcf388cd2bccf8ce0c402c6c605bdfcfa97580be2959"} Sep 06 10:43:56.269479 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:56.269450 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" Sep 06 10:43:56.287481 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:43:56.287422 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" podStartSLOduration=1.287406658 podStartE2EDuration="1.287406658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:43:56.285834928 +0000 UTC m=+829.868324559" watchObservedRunningTime="2026-09-06 10:43:56.287406658 +0000 UTC m=+829.869896289" Sep 06 10:44:06.329582 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:06.329552 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:07.277795 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:07.277708 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" Sep 06 10:44:16.199788 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:16.199750 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:18.191908 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:18.191867 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:21.839047 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:21.839007 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:21.896841 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:21.896783 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:21.978787 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:21.978755 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:22.077605 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:22.077573 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:27.428297 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:27.428265 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:29.980916 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:29.980879 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:37.828833 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:37.828790 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:40.548810 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.548777 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:40.574860 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.574826 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l"] Sep 06 10:44:40.575186 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.575138 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" podUID="63800b4f-2b28-4a38-aee2-e70ca0c07161" containerName="payload-processing" containerID="cri-o://4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65" gracePeriod=30 Sep 06 10:44:40.649049 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.649016 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:40.725560 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.725524 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:40.791385 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.791346 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:40.837021 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.836999 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" Sep 06 10:44:40.891527 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.891499 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm5fs\" (UniqueName: \"kubernetes.io/projected/63800b4f-2b28-4a38-aee2-e70ca0c07161-kube-api-access-hm5fs\") pod \"63800b4f-2b28-4a38-aee2-e70ca0c07161\" (UID: \"63800b4f-2b28-4a38-aee2-e70ca0c07161\") " Sep 06 10:44:40.891693 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.891567 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63800b4f-2b28-4a38-aee2-e70ca0c07161-plugins-config-volume\") pod \"63800b4f-2b28-4a38-aee2-e70ca0c07161\" (UID: \"63800b4f-2b28-4a38-aee2-e70ca0c07161\") " Sep 06 10:44:40.891978 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.891950 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63800b4f-2b28-4a38-aee2-e70ca0c07161-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "63800b4f-2b28-4a38-aee2-e70ca0c07161" (UID: "63800b4f-2b28-4a38-aee2-e70ca0c07161"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:44:40.893539 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.893517 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63800b4f-2b28-4a38-aee2-e70ca0c07161-kube-api-access-hm5fs" (OuterVolumeSpecName: "kube-api-access-hm5fs") pod "63800b4f-2b28-4a38-aee2-e70ca0c07161" (UID: "63800b4f-2b28-4a38-aee2-e70ca0c07161"). InnerVolumeSpecName "kube-api-access-hm5fs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:44:40.992814 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.992781 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm5fs\" (UniqueName: \"kubernetes.io/projected/63800b4f-2b28-4a38-aee2-e70ca0c07161-kube-api-access-hm5fs\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:44:40.992814 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:40.992807 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63800b4f-2b28-4a38-aee2-e70ca0c07161-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:44:41.059525 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:41.059486 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:41.521796 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:41.521758 2574 generic.go:356] "Generic (PLEG): container finished" podID="63800b4f-2b28-4a38-aee2-e70ca0c07161" containerID="4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65" exitCode=0 Sep 06 10:44:41.521969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:41.521802 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" Sep 06 10:44:41.521969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:41.521850 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" event={"ID":"63800b4f-2b28-4a38-aee2-e70ca0c07161","Type":"ContainerDied","Data":"4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65"} Sep 06 10:44:41.521969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:41.521901 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l" event={"ID":"63800b4f-2b28-4a38-aee2-e70ca0c07161","Type":"ContainerDied","Data":"239f961ede3edf3125f7bcf388cd2bccf8ce0c402c6c605bdfcfa97580be2959"} Sep 06 10:44:41.521969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:41.521929 2574 scope.go:117] "RemoveContainer" containerID="4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65" Sep 06 10:44:41.533675 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:41.533653 2574 scope.go:117] "RemoveContainer" containerID="4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65" Sep 06 10:44:41.534026 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:44:41.534004 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65\": container with ID starting with 4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65 not found: ID does not exist" containerID="4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65" Sep 06 10:44:41.534095 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:41.534038 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65"} err="failed to get container status \"4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65\": rpc error: code = NotFound desc = could not find container \"4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65\": container with ID starting with 4c7edac0dfc9fae43d92b8465ecf92efc206b7df1109709fb05425436401dd65 not found: ID does not exist" Sep 06 10:44:41.538018 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:41.537994 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l"] Sep 06 10:44:41.546180 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:41.546156 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ccf833-745684f765-gm48l"] Sep 06 10:44:42.927653 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:42.927614 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63800b4f-2b28-4a38-aee2-e70ca0c07161" path="/var/lib/kubelet/pods/63800b4f-2b28-4a38-aee2-e70ca0c07161/volumes" Sep 06 10:44:43.458476 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:43.458448 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:43.565095 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:43.565064 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:43.660688 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:43.660651 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:46.135387 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:46.135351 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:47.230999 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:47.230967 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:47.332200 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:47.332171 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:47.478396 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:47.478362 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:47.905775 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:47.905744 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:48.201421 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:48.201388 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:48.348951 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:48.348919 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:50.012318 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:50.012272 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:50.112685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:50.112648 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:50.212029 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:50.212000 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:54.128748 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:54.128713 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:54.880554 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:54.880518 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:44:56.721371 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:44:56.721342 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:00.144095 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.144016 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811525-zzzjn"] Sep 06 10:45:00.144450 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.144440 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63800b4f-2b28-4a38-aee2-e70ca0c07161" containerName="payload-processing" Sep 06 10:45:00.144498 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.144453 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="63800b4f-2b28-4a38-aee2-e70ca0c07161" containerName="payload-processing" Sep 06 10:45:00.144545 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.144537 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="63800b4f-2b28-4a38-aee2-e70ca0c07161" containerName="payload-processing" Sep 06 10:45:00.149200 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.149180 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811525-zzzjn" Sep 06 10:45:00.151431 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.151405 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\"" Sep 06 10:45:00.154346 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.154321 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811525-zzzjn"] Sep 06 10:45:00.179068 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.179038 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811525-zzzjn" Sep 06 10:45:00.454834 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.454803 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811525-zzzjn"] Sep 06 10:45:00.455841 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:45:00.455817 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa0dc5d_1798_4c50_a58d_b4db321a42af.slice/crio-a8a26c4fdae1453a4267003ecc1529e9af27a9de5c8d1ad867c6e018ee3c58bd WatchSource:0}: Error finding container a8a26c4fdae1453a4267003ecc1529e9af27a9de5c8d1ad867c6e018ee3c58bd: Status 404 returned error can't find the container with id a8a26c4fdae1453a4267003ecc1529e9af27a9de5c8d1ad867c6e018ee3c58bd Sep 06 10:45:00.457714 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.457688 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 06 10:45:00.632074 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.632038 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811525-zzzjn" event={"ID":"1fa0dc5d-1798-4c50-a58d-b4db321a42af","Type":"ContainerStarted","Data":"a8a26c4fdae1453a4267003ecc1529e9af27a9de5c8d1ad867c6e018ee3c58bd"} Sep 06 10:45:00.669808 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.669769 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:00.774866 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.774788 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:00.846270 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.846236 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:00.912314 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:00.912282 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:01.560327 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:01.560276 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:01.622043 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:01.621986 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:01.685951 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:01.685891 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:01.781667 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:01.781636 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:01.929409 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:01.929372 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:02.124305 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:02.124271 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:03.231434 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:03.231395 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:03.653918 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:03.653816 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811525-zzzjn" event={"ID":"1fa0dc5d-1798-4c50-a58d-b4db321a42af","Type":"ContainerStarted","Data":"b2133296d64ebd3bdd25dffc58922a5338edc3432b86c986eda74a315c9d802a"} Sep 06 10:45:03.668469 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:03.668416 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811525-zzzjn" podStartSLOduration=1.1852865989999999 podStartE2EDuration="3.668400214s" totalImagesPullingTime="2.483113615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:45:03.667736128 +0000 UTC m=+897.250225753" watchObservedRunningTime="2026-09-06 10:45:03.668400214 +0000 UTC m=+897.250889863" Sep 06 10:45:04.661089 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:04.661053 2574 generic.go:356] "Generic (PLEG): container finished" podID="1fa0dc5d-1798-4c50-a58d-b4db321a42af" containerID="b2133296d64ebd3bdd25dffc58922a5338edc3432b86c986eda74a315c9d802a" exitCode=0 Sep 06 10:45:04.661522 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:04.661145 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811525-zzzjn" event={"ID":"1fa0dc5d-1798-4c50-a58d-b4db321a42af","Type":"ContainerDied","Data":"b2133296d64ebd3bdd25dffc58922a5338edc3432b86c986eda74a315c9d802a"} Sep 06 10:45:05.801690 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:05.801665 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811525-zzzjn" Sep 06 10:45:06.673742 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:06.673715 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811525-zzzjn" Sep 06 10:45:06.673914 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:06.673756 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811525-zzzjn" event={"ID":"1fa0dc5d-1798-4c50-a58d-b4db321a42af","Type":"ContainerDied","Data":"a8a26c4fdae1453a4267003ecc1529e9af27a9de5c8d1ad867c6e018ee3c58bd"} Sep 06 10:45:06.673914 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:06.673801 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8a26c4fdae1453a4267003ecc1529e9af27a9de5c8d1ad867c6e018ee3c58bd" Sep 06 10:45:07.117852 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:07.117818 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:09.072564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:09.072529 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:09.129809 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:09.129774 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:09.730459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:09.730421 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:13.301324 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:13.301284 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g"] Sep 06 10:45:13.307774 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:13.307749 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:14.326181 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:14.326146 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:14.643404 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:14.643319 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:15.825460 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:15.825422 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:15.888365 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:15.888337 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:16.051509 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:16.051469 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:16.346636 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:16.346601 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:16.516694 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:16.516652 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:19.789042 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:19.789006 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:19.879664 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:19.879633 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:20.995725 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:20.995691 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:21.096680 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.096646 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:21.372550 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.372472 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:21.474398 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.474357 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:21.575831 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.575792 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:21.604266 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.604232 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb"] Sep 06 10:45:21.604764 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.604741 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fa0dc5d-1798-4c50-a58d-b4db321a42af" containerName="cleanup" Sep 06 10:45:21.604871 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.604766 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa0dc5d-1798-4c50-a58d-b4db321a42af" containerName="cleanup" Sep 06 10:45:21.604929 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.604914 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fa0dc5d-1798-4c50-a58d-b4db321a42af" containerName="cleanup" Sep 06 10:45:21.609527 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.609504 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" Sep 06 10:45:21.611678 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.611653 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2313d276-dockercfg-nk2md\"" Sep 06 10:45:21.611804 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.611785 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2313d276\"" Sep 06 10:45:21.619190 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.619163 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb"] Sep 06 10:45:21.677596 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.677563 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg9tc\" (UniqueName: \"kubernetes.io/projected/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c-kube-api-access-fg9tc\") pod \"payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb\" (UID: \"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c\") " pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" Sep 06 10:45:21.677795 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.677607 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c-plugins-config-volume\") pod \"payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb\" (UID: \"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c\") " pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" Sep 06 10:45:21.779288 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.779253 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg9tc\" (UniqueName: \"kubernetes.io/projected/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c-kube-api-access-fg9tc\") pod \"payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb\" (UID: \"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c\") " pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" Sep 06 10:45:21.779288 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.779293 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c-plugins-config-volume\") pod \"payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb\" (UID: \"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c\") " pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" Sep 06 10:45:21.779995 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.779975 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c-plugins-config-volume\") pod \"payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb\" (UID: \"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c\") " pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" Sep 06 10:45:21.786359 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.786330 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg9tc\" (UniqueName: \"kubernetes.io/projected/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c-kube-api-access-fg9tc\") pod \"payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb\" (UID: \"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c\") " pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" Sep 06 10:45:21.921699 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:21.921667 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" Sep 06 10:45:22.194173 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:22.194145 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb"] Sep 06 10:45:22.196390 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:45:22.196361 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b3b7a51_d0f1_46f7_b74c_c1f6601a9e0c.slice/crio-e0a3151639d0a8878b40ef1b7835896ddcbd625d987eaa3b7571d640b8295a5a WatchSource:0}: Error finding container e0a3151639d0a8878b40ef1b7835896ddcbd625d987eaa3b7571d640b8295a5a: Status 404 returned error can't find the container with id e0a3151639d0a8878b40ef1b7835896ddcbd625d987eaa3b7571d640b8295a5a Sep 06 10:45:22.764335 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:22.764297 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" event={"ID":"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c","Type":"ContainerStarted","Data":"c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964"} Sep 06 10:45:22.764526 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:22.764392 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" Sep 06 10:45:22.764526 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:22.764415 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" event={"ID":"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c","Type":"ContainerStarted","Data":"e0a3151639d0a8878b40ef1b7835896ddcbd625d987eaa3b7571d640b8295a5a"} Sep 06 10:45:22.783058 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:22.783004 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" podStartSLOduration=1.782987882 podStartE2EDuration="1.782987882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:45:22.779537635 +0000 UTC m=+916.362027266" watchObservedRunningTime="2026-09-06 10:45:22.782987882 +0000 UTC m=+916.365477514" Sep 06 10:45:24.541574 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:24.541530 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:25.108519 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.108478 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:25.138848 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.138817 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr"] Sep 06 10:45:25.139090 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.139045 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" podUID="86dae43f-0771-497d-bd80-ee09793cfe39" containerName="payload-processing" containerID="cri-o://e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3" gracePeriod=30 Sep 06 10:45:25.147801 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.147769 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz"] Sep 06 10:45:25.148224 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.148023 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" podUID="c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3" containerName="payload-pre-processing" containerID="cri-o://9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9" gracePeriod=30 Sep 06 10:45:25.206880 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.206847 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:25.307740 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.307707 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:25.406932 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.406889 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:25.430451 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.430425 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" Sep 06 10:45:25.433039 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.433020 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" Sep 06 10:45:25.515873 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.515841 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86dae43f-0771-497d-bd80-ee09793cfe39-plugins-config-volume\") pod \"86dae43f-0771-497d-bd80-ee09793cfe39\" (UID: \"86dae43f-0771-497d-bd80-ee09793cfe39\") " Sep 06 10:45:25.516034 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.515894 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3-plugins-config-volume\") pod \"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3\" (UID: \"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3\") " Sep 06 10:45:25.516034 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.515929 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw2nj\" (UniqueName: \"kubernetes.io/projected/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3-kube-api-access-mw2nj\") pod \"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3\" (UID: \"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3\") " Sep 06 10:45:25.516034 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.516001 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtcw4\" (UniqueName: \"kubernetes.io/projected/86dae43f-0771-497d-bd80-ee09793cfe39-kube-api-access-rtcw4\") pod \"86dae43f-0771-497d-bd80-ee09793cfe39\" (UID: \"86dae43f-0771-497d-bd80-ee09793cfe39\") " Sep 06 10:45:25.516266 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.516237 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86dae43f-0771-497d-bd80-ee09793cfe39-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "86dae43f-0771-497d-bd80-ee09793cfe39" (UID: "86dae43f-0771-497d-bd80-ee09793cfe39"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:45:25.516324 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.516298 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3" (UID: "c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:45:25.516369 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.516340 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86dae43f-0771-497d-bd80-ee09793cfe39-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:25.518116 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.518093 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3-kube-api-access-mw2nj" (OuterVolumeSpecName: "kube-api-access-mw2nj") pod "c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3" (UID: "c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3"). InnerVolumeSpecName "kube-api-access-mw2nj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:45:25.518230 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.518117 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86dae43f-0771-497d-bd80-ee09793cfe39-kube-api-access-rtcw4" (OuterVolumeSpecName: "kube-api-access-rtcw4") pod "86dae43f-0771-497d-bd80-ee09793cfe39" (UID: "86dae43f-0771-497d-bd80-ee09793cfe39"). InnerVolumeSpecName "kube-api-access-rtcw4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:45:25.617201 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.617171 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtcw4\" (UniqueName: \"kubernetes.io/projected/86dae43f-0771-497d-bd80-ee09793cfe39-kube-api-access-rtcw4\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:25.617201 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.617195 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:25.617201 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.617204 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw2nj\" (UniqueName: \"kubernetes.io/projected/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3-kube-api-access-mw2nj\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:25.669572 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.669524 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:25.785551 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.785518 2574 generic.go:356] "Generic (PLEG): container finished" podID="86dae43f-0771-497d-bd80-ee09793cfe39" containerID="e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3" exitCode=0 Sep 06 10:45:25.785736 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.785577 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" Sep 06 10:45:25.785736 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.785601 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" event={"ID":"86dae43f-0771-497d-bd80-ee09793cfe39","Type":"ContainerDied","Data":"e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3"} Sep 06 10:45:25.785736 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.785669 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr" event={"ID":"86dae43f-0771-497d-bd80-ee09793cfe39","Type":"ContainerDied","Data":"fa06430e8f3014b4092f107fc33e77840e8d5c0835b776366f7437011df27ce0"} Sep 06 10:45:25.785736 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.785702 2574 scope.go:117] "RemoveContainer" containerID="e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3" Sep 06 10:45:25.787875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.787818 2574 generic.go:356] "Generic (PLEG): container finished" podID="c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3" containerID="9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9" exitCode=0 Sep 06 10:45:25.787875 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.787865 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" Sep 06 10:45:25.788042 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.787868 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" event={"ID":"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3","Type":"ContainerDied","Data":"9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9"} Sep 06 10:45:25.788042 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.787936 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz" event={"ID":"c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3","Type":"ContainerDied","Data":"466f6542918ff0774838f59a637d2995d6005d3980dcf4b20b7a89a99459de5b"} Sep 06 10:45:25.797374 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.797350 2574 scope.go:117] "RemoveContainer" containerID="e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3" Sep 06 10:45:25.797677 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:45:25.797655 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3\": container with ID starting with e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3 not found: ID does not exist" containerID="e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3" Sep 06 10:45:25.797777 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.797688 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3"} err="failed to get container status \"e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3\": rpc error: code = NotFound desc = could not find container \"e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3\": container with ID starting with e17ea26af17da54e0c280644d93d628d064e41fe5e2076b5059ede300b4c36e3 not found: ID does not exist" Sep 06 10:45:25.797777 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.797710 2574 scope.go:117] "RemoveContainer" containerID="9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9" Sep 06 10:45:25.810438 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.810409 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr"] Sep 06 10:45:25.813529 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.813506 2574 scope.go:117] "RemoveContainer" containerID="9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9" Sep 06 10:45:25.813813 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.813795 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-232aeb-6fd4cccb55-p8qfr"] Sep 06 10:45:25.813892 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:45:25.813821 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9\": container with ID starting with 9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9 not found: ID does not exist" containerID="9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9" Sep 06 10:45:25.813892 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.813847 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9"} err="failed to get container status \"9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9\": rpc error: code = NotFound desc = could not find container \"9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9\": container with ID starting with 9d81f60cafad17bd1681e7e320fdf5435e4cc76bce445bc4ec5d14ae409dc2f9 not found: ID does not exist" Sep 06 10:45:25.825527 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.825504 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz"] Sep 06 10:45:25.828785 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:25.828762 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-232aeb-7c6744dcdb-c9zvz"] Sep 06 10:45:26.926239 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:26.926206 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86dae43f-0771-497d-bd80-ee09793cfe39" path="/var/lib/kubelet/pods/86dae43f-0771-497d-bd80-ee09793cfe39/volumes" Sep 06 10:45:26.926605 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:26.926564 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3" path="/var/lib/kubelet/pods/c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3/volumes" Sep 06 10:45:28.321298 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.321244 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" podUID="ff67ccc1-a644-4940-891d-e2ee689c1c54" containerName="main" containerID="cri-o://d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd" gracePeriod=45 Sep 06 10:45:28.582506 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.582449 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:45:28.646770 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.646740 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-dshm\") pod \"ff67ccc1-a644-4940-891d-e2ee689c1c54\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " Sep 06 10:45:28.646926 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.646777 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff67ccc1-a644-4940-891d-e2ee689c1c54-tls-certs\") pod \"ff67ccc1-a644-4940-891d-e2ee689c1c54\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " Sep 06 10:45:28.646926 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.646797 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-home\") pod \"ff67ccc1-a644-4940-891d-e2ee689c1c54\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " Sep 06 10:45:28.646926 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.646877 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-model-cache\") pod \"ff67ccc1-a644-4940-891d-e2ee689c1c54\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " Sep 06 10:45:28.646926 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.646919 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-tmp-dir\") pod \"ff67ccc1-a644-4940-891d-e2ee689c1c54\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " Sep 06 10:45:28.647274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.646939 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fv4h\" (UniqueName: \"kubernetes.io/projected/ff67ccc1-a644-4940-891d-e2ee689c1c54-kube-api-access-5fv4h\") pod \"ff67ccc1-a644-4940-891d-e2ee689c1c54\" (UID: \"ff67ccc1-a644-4940-891d-e2ee689c1c54\") " Sep 06 10:45:28.647274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.647080 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-home" (OuterVolumeSpecName: "home") pod "ff67ccc1-a644-4940-891d-e2ee689c1c54" (UID: "ff67ccc1-a644-4940-891d-e2ee689c1c54"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:45:28.647274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.647148 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff67ccc1-a644-4940-891d-e2ee689c1c54" (UID: "ff67ccc1-a644-4940-891d-e2ee689c1c54"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:45:28.647274 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.647204 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff67ccc1-a644-4940-891d-e2ee689c1c54" (UID: "ff67ccc1-a644-4940-891d-e2ee689c1c54"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:45:28.647418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.647310 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-home\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:28.647418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.647326 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-model-cache\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:28.647418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.647339 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-tmp-dir\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:28.649099 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.649077 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff67ccc1-a644-4940-891d-e2ee689c1c54-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff67ccc1-a644-4940-891d-e2ee689c1c54" (UID: "ff67ccc1-a644-4940-891d-e2ee689c1c54"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:45:28.649210 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.649143 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff67ccc1-a644-4940-891d-e2ee689c1c54-kube-api-access-5fv4h" (OuterVolumeSpecName: "kube-api-access-5fv4h") pod "ff67ccc1-a644-4940-891d-e2ee689c1c54" (UID: "ff67ccc1-a644-4940-891d-e2ee689c1c54"). InnerVolumeSpecName "kube-api-access-5fv4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:45:28.649210 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.649165 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-dshm" (OuterVolumeSpecName: "dshm") pod "ff67ccc1-a644-4940-891d-e2ee689c1c54" (UID: "ff67ccc1-a644-4940-891d-e2ee689c1c54"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:45:28.748789 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.748752 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff67ccc1-a644-4940-891d-e2ee689c1c54-dshm\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:28.748789 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.748783 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff67ccc1-a644-4940-891d-e2ee689c1c54-tls-certs\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:28.748789 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.748793 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fv4h\" (UniqueName: \"kubernetes.io/projected/ff67ccc1-a644-4940-891d-e2ee689c1c54-kube-api-access-5fv4h\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:28.807646 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.807608 2574 generic.go:356] "Generic (PLEG): container finished" podID="ff67ccc1-a644-4940-891d-e2ee689c1c54" containerID="d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd" exitCode=0 Sep 06 10:45:28.807814 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.807653 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" Sep 06 10:45:28.807814 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.807701 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" event={"ID":"ff67ccc1-a644-4940-891d-e2ee689c1c54","Type":"ContainerDied","Data":"d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd"} Sep 06 10:45:28.807814 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.807748 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g" event={"ID":"ff67ccc1-a644-4940-891d-e2ee689c1c54","Type":"ContainerDied","Data":"6899f05c8ba01f4454dcba6379a065b84c37e541f9510cd407c458f6dcb1cc67"} Sep 06 10:45:28.807814 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.807764 2574 scope.go:117] "RemoveContainer" containerID="d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd" Sep 06 10:45:28.817028 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.817009 2574 scope.go:117] "RemoveContainer" containerID="d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd" Sep 06 10:45:28.817336 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:45:28.817316 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd\": container with ID starting with d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd not found: ID does not exist" containerID="d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd" Sep 06 10:45:28.817403 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.817348 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd"} err="failed to get container status \"d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd\": rpc error: code = NotFound desc = could not find container \"d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd\": container with ID starting with d8cf6dac54b166ee90c7c2d4a31414d63893919db8687ad568f997d1030b79fd not found: ID does not exist" Sep 06 10:45:28.827816 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.827792 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g"] Sep 06 10:45:28.832370 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.832347 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-232aeb/test-model-232aeb-kserve-7bfd5766dc-2sr2g"] Sep 06 10:45:28.926424 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:28.926396 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff67ccc1-a644-4940-891d-e2ee689c1c54" path="/var/lib/kubelet/pods/ff67ccc1-a644-4940-891d-e2ee689c1c54/volumes" Sep 06 10:45:31.540339 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:31.540295 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:31.609368 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:31.609334 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:31.767081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:31.767044 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz"] Sep 06 10:45:31.767305 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:31.767278 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" containerID="cri-o://0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802" gracePeriod=30 Sep 06 10:45:31.787744 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:31.787714 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:32.365202 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:32.365167 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:33.772637 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:33.772595 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" Sep 06 10:45:37.013706 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.013681 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:45:37.127158 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127053 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drlhr\" (UniqueName: \"kubernetes.io/projected/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-kube-api-access-drlhr\") pod \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " Sep 06 10:45:37.127158 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127103 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-data\") pod \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " Sep 06 10:45:37.127386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127181 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-credential-socket\") pod \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " Sep 06 10:45:37.127386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127200 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-token\") pod \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " Sep 06 10:45:37.127386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127238 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-envoy\") pod \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " Sep 06 10:45:37.127386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127277 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-podinfo\") pod \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " Sep 06 10:45:37.127386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127334 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-workload-socket\") pod \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " Sep 06 10:45:37.127386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127364 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istiod-ca-cert\") pod \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " Sep 06 10:45:37.127698 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127404 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-workload-certs\") pod \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\" (UID: \"ad17d760-53d2-4cb2-afb7-e6eb810a4b18\") " Sep 06 10:45:37.127698 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127502 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ad17d760-53d2-4cb2-afb7-e6eb810a4b18" (UID: "ad17d760-53d2-4cb2-afb7-e6eb810a4b18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:45:37.127698 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127528 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-data" (OuterVolumeSpecName: "istio-data") pod "ad17d760-53d2-4cb2-afb7-e6eb810a4b18" (UID: "ad17d760-53d2-4cb2-afb7-e6eb810a4b18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:45:37.127698 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127618 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ad17d760-53d2-4cb2-afb7-e6eb810a4b18" (UID: "ad17d760-53d2-4cb2-afb7-e6eb810a4b18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:45:37.127910 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127708 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ad17d760-53d2-4cb2-afb7-e6eb810a4b18" (UID: "ad17d760-53d2-4cb2-afb7-e6eb810a4b18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:45:37.127910 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127815 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ad17d760-53d2-4cb2-afb7-e6eb810a4b18" (UID: "ad17d760-53d2-4cb2-afb7-e6eb810a4b18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:45:37.127910 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127892 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-credential-socket\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:37.128035 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127913 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-workload-socket\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:37.128035 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127928 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-workload-certs\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:37.128035 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.127941 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-data\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:37.129431 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.129404 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-token" (OuterVolumeSpecName: "istio-token") pod "ad17d760-53d2-4cb2-afb7-e6eb810a4b18" (UID: "ad17d760-53d2-4cb2-afb7-e6eb810a4b18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:45:37.129619 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.129597 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ad17d760-53d2-4cb2-afb7-e6eb810a4b18" (UID: "ad17d760-53d2-4cb2-afb7-e6eb810a4b18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 06 10:45:37.129782 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.129762 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ad17d760-53d2-4cb2-afb7-e6eb810a4b18" (UID: "ad17d760-53d2-4cb2-afb7-e6eb810a4b18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:45:37.129782 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.129768 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-kube-api-access-drlhr" (OuterVolumeSpecName: "kube-api-access-drlhr") pod "ad17d760-53d2-4cb2-afb7-e6eb810a4b18" (UID: "ad17d760-53d2-4cb2-afb7-e6eb810a4b18"). InnerVolumeSpecName "kube-api-access-drlhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:45:37.229364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.229337 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istiod-ca-cert\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:37.229364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.229360 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drlhr\" (UniqueName: \"kubernetes.io/projected/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-kube-api-access-drlhr\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:37.229364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.229369 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-token\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:37.229579 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.229377 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-envoy\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:37.229579 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.229385 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad17d760-53d2-4cb2-afb7-e6eb810a4b18-istio-podinfo\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:37.862143 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.862090 2574 generic.go:356] "Generic (PLEG): container finished" podID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerID="0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802" exitCode=0 Sep 06 10:45:37.862339 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.862156 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" Sep 06 10:45:37.862339 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.862179 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" event={"ID":"ad17d760-53d2-4cb2-afb7-e6eb810a4b18","Type":"ContainerDied","Data":"0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802"} Sep 06 10:45:37.862339 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.862240 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz" event={"ID":"ad17d760-53d2-4cb2-afb7-e6eb810a4b18","Type":"ContainerDied","Data":"d3f560573f9decd06c25b7c7dae3f4e234b143ac2e4d3fc9ab5ebada85fb93fc"} Sep 06 10:45:37.862339 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.862269 2574 scope.go:117] "RemoveContainer" containerID="0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802" Sep 06 10:45:37.872220 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.872202 2574 scope.go:117] "RemoveContainer" containerID="492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503" Sep 06 10:45:37.880059 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.880040 2574 scope.go:117] "RemoveContainer" containerID="0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802" Sep 06 10:45:37.880326 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:45:37.880308 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802\": container with ID starting with 0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802 not found: ID does not exist" containerID="0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802" Sep 06 10:45:37.880377 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.880334 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802"} err="failed to get container status \"0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802\": rpc error: code = NotFound desc = could not find container \"0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802\": container with ID starting with 0ed93dc17e4031e24e702a70e6ff953f55e10ca7b30acd6035da2a45496b6802 not found: ID does not exist" Sep 06 10:45:37.880377 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.880350 2574 scope.go:117] "RemoveContainer" containerID="492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503" Sep 06 10:45:37.880583 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:45:37.880569 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503\": container with ID starting with 492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503 not found: ID does not exist" containerID="492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503" Sep 06 10:45:37.880623 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.880587 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503"} err="failed to get container status \"492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503\": rpc error: code = NotFound desc = could not find container \"492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503\": container with ID starting with 492cd39a5b1d843955731d809b056a9854161ae90118f50506bb32c3b36ff503 not found: ID does not exist" Sep 06 10:45:37.884768 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.884746 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz"] Sep 06 10:45:37.887064 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:37.887044 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-232aeb-openshift-default-6d9ffcb9b-b8qfz"] Sep 06 10:45:38.927334 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:38.927300 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" path="/var/lib/kubelet/pods/ad17d760-53d2-4cb2-afb7-e6eb810a4b18/volumes" Sep 06 10:45:41.936783 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:41.936751 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:42.480896 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.480864 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:42.503063 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.503026 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc"] Sep 06 10:45:42.503286 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.503240 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" podUID="b5ab2ec7-331b-4b12-8aba-dce4ce479db0" containerName="payload-processing" containerID="cri-o://52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294" gracePeriod=30 Sep 06 10:45:42.517633 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.517605 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg"] Sep 06 10:45:42.518539 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.517844 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" podUID="3c5e4dcd-acee-424f-b57c-45c5e547f70e" containerName="payload-pre-processing" containerID="cri-o://765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4" gracePeriod=30 Sep 06 10:45:42.580051 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.580022 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:42.661136 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.661077 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:42.718529 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.718494 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:42.789914 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.789890 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" Sep 06 10:45:42.792800 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.792780 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" Sep 06 10:45:42.881800 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.881765 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8xh2\" (UniqueName: \"kubernetes.io/projected/b5ab2ec7-331b-4b12-8aba-dce4ce479db0-kube-api-access-t8xh2\") pod \"b5ab2ec7-331b-4b12-8aba-dce4ce479db0\" (UID: \"b5ab2ec7-331b-4b12-8aba-dce4ce479db0\") " Sep 06 10:45:42.883853 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.883827 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5ab2ec7-331b-4b12-8aba-dce4ce479db0-kube-api-access-t8xh2" (OuterVolumeSpecName: "kube-api-access-t8xh2") pod "b5ab2ec7-331b-4b12-8aba-dce4ce479db0" (UID: "b5ab2ec7-331b-4b12-8aba-dce4ce479db0"). InnerVolumeSpecName "kube-api-access-t8xh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:45:42.895172 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.895142 2574 generic.go:356] "Generic (PLEG): container finished" podID="3c5e4dcd-acee-424f-b57c-45c5e547f70e" containerID="765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4" exitCode=0 Sep 06 10:45:42.895297 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.895193 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" Sep 06 10:45:42.895297 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.895216 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" event={"ID":"3c5e4dcd-acee-424f-b57c-45c5e547f70e","Type":"ContainerDied","Data":"765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4"} Sep 06 10:45:42.895297 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.895269 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg" event={"ID":"3c5e4dcd-acee-424f-b57c-45c5e547f70e","Type":"ContainerDied","Data":"f98a4258eece0ad28ce249458840d2a1616bd35969f6a8464d0ab52f63f94f7c"} Sep 06 10:45:42.895424 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.895299 2574 scope.go:117] "RemoveContainer" containerID="765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4" Sep 06 10:45:42.896621 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.896602 2574 generic.go:356] "Generic (PLEG): container finished" podID="b5ab2ec7-331b-4b12-8aba-dce4ce479db0" containerID="52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294" exitCode=0 Sep 06 10:45:42.896704 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.896646 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" Sep 06 10:45:42.896704 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.896689 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" event={"ID":"b5ab2ec7-331b-4b12-8aba-dce4ce479db0","Type":"ContainerDied","Data":"52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294"} Sep 06 10:45:42.896794 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.896742 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc" event={"ID":"b5ab2ec7-331b-4b12-8aba-dce4ce479db0","Type":"ContainerDied","Data":"38ead67a6101240db2c2b78295df73c0e406b458ff57d66cde899aa3affd506e"} Sep 06 10:45:42.904037 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.904019 2574 scope.go:117] "RemoveContainer" containerID="765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4" Sep 06 10:45:42.904310 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:45:42.904289 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4\": container with ID starting with 765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4 not found: ID does not exist" containerID="765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4" Sep 06 10:45:42.904384 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.904317 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4"} err="failed to get container status \"765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4\": rpc error: code = NotFound desc = could not find container \"765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4\": container with ID starting with 765a60604f4c5562350a62d5ddbe2af8ee1fe110857febb1931360722fdda6e4 not found: ID does not exist" Sep 06 10:45:42.904384 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.904333 2574 scope.go:117] "RemoveContainer" containerID="52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294" Sep 06 10:45:42.911985 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.911969 2574 scope.go:117] "RemoveContainer" containerID="52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294" Sep 06 10:45:42.912274 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:45:42.912251 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294\": container with ID starting with 52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294 not found: ID does not exist" containerID="52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294" Sep 06 10:45:42.912341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.912275 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294"} err="failed to get container status \"52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294\": rpc error: code = NotFound desc = could not find container \"52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294\": container with ID starting with 52f4f4038f4740ef570d89509d2f7c3bd40e41a8532b87880980d8463b1fe294 not found: ID does not exist" Sep 06 10:45:42.983254 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.983223 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c5e4dcd-acee-424f-b57c-45c5e547f70e-plugins-config-volume\") pod \"3c5e4dcd-acee-424f-b57c-45c5e547f70e\" (UID: \"3c5e4dcd-acee-424f-b57c-45c5e547f70e\") " Sep 06 10:45:42.983254 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.983257 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz2ch\" (UniqueName: \"kubernetes.io/projected/3c5e4dcd-acee-424f-b57c-45c5e547f70e-kube-api-access-pz2ch\") pod \"3c5e4dcd-acee-424f-b57c-45c5e547f70e\" (UID: \"3c5e4dcd-acee-424f-b57c-45c5e547f70e\") " Sep 06 10:45:42.983749 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.983283 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ab2ec7-331b-4b12-8aba-dce4ce479db0-plugins-config-volume\") pod \"b5ab2ec7-331b-4b12-8aba-dce4ce479db0\" (UID: \"b5ab2ec7-331b-4b12-8aba-dce4ce479db0\") " Sep 06 10:45:42.983749 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.983577 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8xh2\" (UniqueName: \"kubernetes.io/projected/b5ab2ec7-331b-4b12-8aba-dce4ce479db0-kube-api-access-t8xh2\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:42.983749 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.983649 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c5e4dcd-acee-424f-b57c-45c5e547f70e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3c5e4dcd-acee-424f-b57c-45c5e547f70e" (UID: "3c5e4dcd-acee-424f-b57c-45c5e547f70e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:45:42.983749 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.983731 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5ab2ec7-331b-4b12-8aba-dce4ce479db0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5ab2ec7-331b-4b12-8aba-dce4ce479db0" (UID: "b5ab2ec7-331b-4b12-8aba-dce4ce479db0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:45:42.985480 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:42.985451 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c5e4dcd-acee-424f-b57c-45c5e547f70e-kube-api-access-pz2ch" (OuterVolumeSpecName: "kube-api-access-pz2ch") pod "3c5e4dcd-acee-424f-b57c-45c5e547f70e" (UID: "3c5e4dcd-acee-424f-b57c-45c5e547f70e"). InnerVolumeSpecName "kube-api-access-pz2ch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:45:43.048710 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:43.048619 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:43.084754 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:43.084724 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c5e4dcd-acee-424f-b57c-45c5e547f70e-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:43.084754 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:43.084753 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz2ch\" (UniqueName: \"kubernetes.io/projected/3c5e4dcd-acee-424f-b57c-45c5e547f70e-kube-api-access-pz2ch\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:43.084937 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:43.084764 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5ab2ec7-331b-4b12-8aba-dce4ce479db0-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:43.225221 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:43.225185 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg"] Sep 06 10:45:43.228787 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:43.228756 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-636b04-c87d67c94-h9plg"] Sep 06 10:45:43.239774 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:43.239746 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc"] Sep 06 10:45:43.241872 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:43.241849 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-636b04-dbf4b9ff7-rjbxc"] Sep 06 10:45:43.982552 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:43.982516 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:44.739266 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:44.739235 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:44.926388 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:44.926354 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c5e4dcd-acee-424f-b57c-45c5e547f70e" path="/var/lib/kubelet/pods/3c5e4dcd-acee-424f-b57c-45c5e547f70e/volumes" Sep 06 10:45:44.926751 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:44.926738 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5ab2ec7-331b-4b12-8aba-dce4ce479db0" path="/var/lib/kubelet/pods/b5ab2ec7-331b-4b12-8aba-dce4ce479db0/volumes" Sep 06 10:45:45.369983 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:45.369940 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:45.478296 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:45.478267 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:45.791257 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:45.791224 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:48.778840 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:48.778807 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:48.833216 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:48.833152 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:49.079934 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.079845 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:49.571222 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.571183 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:49.843117 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843038 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5"] Sep 06 10:45:49.843530 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843481 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3" containerName="payload-pre-processing" Sep 06 10:45:49.843530 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843495 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3" containerName="payload-pre-processing" Sep 06 10:45:49.843530 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843502 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5ab2ec7-331b-4b12-8aba-dce4ce479db0" containerName="payload-processing" Sep 06 10:45:49.843530 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843510 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5ab2ec7-331b-4b12-8aba-dce4ce479db0" containerName="payload-processing" Sep 06 10:45:49.843530 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843529 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86dae43f-0771-497d-bd80-ee09793cfe39" containerName="payload-processing" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843544 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="86dae43f-0771-497d-bd80-ee09793cfe39" containerName="payload-processing" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843558 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843563 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843570 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843575 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843581 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c5e4dcd-acee-424f-b57c-45c5e547f70e" containerName="payload-pre-processing" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843586 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c5e4dcd-acee-424f-b57c-45c5e547f70e" containerName="payload-pre-processing" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843592 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff67ccc1-a644-4940-891d-e2ee689c1c54" containerName="main" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843605 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff67ccc1-a644-4940-891d-e2ee689c1c54" containerName="main" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843662 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5ab2ec7-331b-4b12-8aba-dce4ce479db0" containerName="payload-processing" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843669 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff67ccc1-a644-4940-891d-e2ee689c1c54" containerName="main" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843678 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="86dae43f-0771-497d-bd80-ee09793cfe39" containerName="payload-processing" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843684 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843690 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad17d760-53d2-4cb2-afb7-e6eb810a4b18" containerName="istio-proxy" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843697 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3dd3055-ccf9-4ba6-99bb-9e7003fa6ed3" containerName="payload-pre-processing" Sep 06 10:45:49.843700 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.843702 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c5e4dcd-acee-424f-b57c-45c5e547f70e" containerName="payload-pre-processing" Sep 06 10:45:49.848631 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.848609 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:49.851071 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.851048 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-2eb161-openshift-default-dockercfg-5qfzt\"" Sep 06 10:45:49.858391 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.858366 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5"] Sep 06 10:45:49.868954 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.868923 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:49.947550 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.947514 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-data\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:49.947794 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.947556 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istiod-ca-cert\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:49.947794 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.947610 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-workload-certs\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:49.947794 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.947678 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-credential-socket\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:49.947794 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.947724 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-workload-socket\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:49.947794 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.947751 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnlvx\" (UniqueName: \"kubernetes.io/projected/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-kube-api-access-tnlvx\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:49.948167 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.947797 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-envoy\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:49.948167 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.947841 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-token\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:49.948167 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:49.947893 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-podinfo\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.048840 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.048805 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-data\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.048840 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.048844 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istiod-ca-cert\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.049080 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.048862 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-workload-certs\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.049080 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.048921 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-credential-socket\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.049080 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.048946 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-workload-socket\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.049080 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.048971 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnlvx\" (UniqueName: \"kubernetes.io/projected/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-kube-api-access-tnlvx\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.049080 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.049009 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-envoy\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.049080 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.049050 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-token\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.049417 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.049090 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-podinfo\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.049417 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.049292 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-credential-socket\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.049417 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.049325 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-workload-certs\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.049554 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.049465 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-workload-socket\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.049554 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.049490 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-data\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.049801 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.049777 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istiod-ca-cert\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.051281 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.051261 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-envoy\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.051534 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.051517 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-podinfo\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.056116 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.056093 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-token\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.056493 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.056473 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnlvx\" (UniqueName: \"kubernetes.io/projected/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-kube-api-access-tnlvx\") pod \"e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.129163 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.129059 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:50.159154 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.159091 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:50.439899 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.439875 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5"] Sep 06 10:45:50.441767 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:45:50.441731 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod045c7a18_e0f4_4e90_945c_d6512aa7d1f4.slice/crio-5e6d02651cd6fc20e78418340fca2fdfc3ff2156f2488f878c5c8fb81727dd74 WatchSource:0}: Error finding container 5e6d02651cd6fc20e78418340fca2fdfc3ff2156f2488f878c5c8fb81727dd74: Status 404 returned error can't find the container with id 5e6d02651cd6fc20e78418340fca2fdfc3ff2156f2488f878c5c8fb81727dd74 Sep 06 10:45:50.443914 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.443883 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:45:50.443999 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.443958 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:45:50.443999 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.443989 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:45:50.951268 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.951227 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" event={"ID":"045c7a18-e0f4-4e90-945c-d6512aa7d1f4","Type":"ContainerStarted","Data":"e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e"} Sep 06 10:45:50.951721 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.951278 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" event={"ID":"045c7a18-e0f4-4e90-945c-d6512aa7d1f4","Type":"ContainerStarted","Data":"5e6d02651cd6fc20e78418340fca2fdfc3ff2156f2488f878c5c8fb81727dd74"} Sep 06 10:45:50.970627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:50.970569 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" podStartSLOduration=1.970551913 podStartE2EDuration="1.970551913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:45:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:45:50.969312632 +0000 UTC m=+944.551802288" watchObservedRunningTime="2026-09-06 10:45:50.970551913 +0000 UTC m=+944.553041543" Sep 06 10:45:51.159665 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:51.159633 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:51.164773 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:51.164745 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:51.536416 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:51.536375 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:51.633525 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:51.633482 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:51.872046 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:51.871965 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh"] Sep 06 10:45:51.875485 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:51.875469 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh" Sep 06 10:45:51.877785 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:51.877761 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 10:45:51.877977 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:51.877855 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\"" Sep 06 10:45:51.882143 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:51.882105 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh"] Sep 06 10:45:51.957218 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:51.957179 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:51.958259 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:51.958240 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:45:51.969503 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:51.969475 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e466c20d-453b-4c70-94f2-3265324c2444-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2313d276-5mhnh\" (UID: \"e466c20d-453b-4c70-94f2-3265324c2444\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh" Sep 06 10:45:52.070807 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:52.070767 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e466c20d-453b-4c70-94f2-3265324c2444-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2313d276-5mhnh\" (UID: \"e466c20d-453b-4c70-94f2-3265324c2444\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh" Sep 06 10:45:52.071360 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:52.071341 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e466c20d-453b-4c70-94f2-3265324c2444-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2313d276-5mhnh\" (UID: \"e466c20d-453b-4c70-94f2-3265324c2444\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh" Sep 06 10:45:52.187599 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:52.187563 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh" Sep 06 10:45:52.457913 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:52.457883 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh"] Sep 06 10:45:52.459078 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:45:52.459052 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode466c20d_453b_4c70_94f2_3265324c2444.slice/crio-744c69bff06abccaaa1a3868ee575676c6bd5775bfc95a4d21c6e170f40faacc WatchSource:0}: Error finding container 744c69bff06abccaaa1a3868ee575676c6bd5775bfc95a4d21c6e170f40faacc: Status 404 returned error can't find the container with id 744c69bff06abccaaa1a3868ee575676c6bd5775bfc95a4d21c6e170f40faacc Sep 06 10:45:52.848361 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:52.848332 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:52.947768 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:52.947735 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:52.963582 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:52.963551 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh" event={"ID":"e466c20d-453b-4c70-94f2-3265324c2444","Type":"ContainerStarted","Data":"9c89bd483e739ce6596f1891b1970b9d31e823b01859db4c31b41e03765f97f4"} Sep 06 10:45:52.963946 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:52.963603 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh" event={"ID":"e466c20d-453b-4c70-94f2-3265324c2444","Type":"ContainerStarted","Data":"744c69bff06abccaaa1a3868ee575676c6bd5775bfc95a4d21c6e170f40faacc"} Sep 06 10:45:52.979392 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:52.979347 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh" podStartSLOduration=1.9793324110000001 podStartE2EDuration="1.979332411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:45:52.976408987 +0000 UTC m=+946.558898617" watchObservedRunningTime="2026-09-06 10:45:52.979332411 +0000 UTC m=+946.561822042" Sep 06 10:45:53.970131 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:53.970098 2574 generic.go:356] "Generic (PLEG): container finished" podID="e466c20d-453b-4c70-94f2-3265324c2444" containerID="9c89bd483e739ce6596f1891b1970b9d31e823b01859db4c31b41e03765f97f4" exitCode=0 Sep 06 10:45:53.970546 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:53.970180 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh" event={"ID":"e466c20d-453b-4c70-94f2-3265324c2444","Type":"ContainerDied","Data":"9c89bd483e739ce6596f1891b1970b9d31e823b01859db4c31b41e03765f97f4"} Sep 06 10:45:54.057142 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.057090 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:54.118554 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.118527 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:54.242564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.242478 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:54.555744 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.555666 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:54.622440 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.622406 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:45:54.702212 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.702179 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r"] Sep 06 10:45:54.705592 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.705575 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" Sep 06 10:45:54.707724 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.707698 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-2eb161\"" Sep 06 10:45:54.707724 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.707718 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-2eb161-dockercfg-kzk2j\"" Sep 06 10:45:54.715644 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.715621 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r"] Sep 06 10:45:54.798035 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.797996 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce904edc-a4bc-4738-aa3c-80de703e9790-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r\" (UID: \"ce904edc-a4bc-4738-aa3c-80de703e9790\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" Sep 06 10:45:54.798035 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.798049 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjk4d\" (UniqueName: \"kubernetes.io/projected/ce904edc-a4bc-4738-aa3c-80de703e9790-kube-api-access-vjk4d\") pod \"payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r\" (UID: \"ce904edc-a4bc-4738-aa3c-80de703e9790\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" Sep 06 10:45:54.899157 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.899036 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce904edc-a4bc-4738-aa3c-80de703e9790-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r\" (UID: \"ce904edc-a4bc-4738-aa3c-80de703e9790\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" Sep 06 10:45:54.899157 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.899096 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjk4d\" (UniqueName: \"kubernetes.io/projected/ce904edc-a4bc-4738-aa3c-80de703e9790-kube-api-access-vjk4d\") pod \"payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r\" (UID: \"ce904edc-a4bc-4738-aa3c-80de703e9790\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" Sep 06 10:45:54.899757 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.899736 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce904edc-a4bc-4738-aa3c-80de703e9790-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r\" (UID: \"ce904edc-a4bc-4738-aa3c-80de703e9790\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" Sep 06 10:45:54.906818 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:54.906786 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjk4d\" (UniqueName: \"kubernetes.io/projected/ce904edc-a4bc-4738-aa3c-80de703e9790-kube-api-access-vjk4d\") pod \"payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r\" (UID: \"ce904edc-a4bc-4738-aa3c-80de703e9790\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" Sep 06 10:45:55.016872 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.016833 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" Sep 06 10:45:55.111523 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.111498 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh" Sep 06 10:45:55.202160 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.202110 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e466c20d-453b-4c70-94f2-3265324c2444-maas-api-service-ca\") pod \"e466c20d-453b-4c70-94f2-3265324c2444\" (UID: \"e466c20d-453b-4c70-94f2-3265324c2444\") " Sep 06 10:45:55.202531 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.202508 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e466c20d-453b-4c70-94f2-3265324c2444-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e466c20d-453b-4c70-94f2-3265324c2444" (UID: "e466c20d-453b-4c70-94f2-3265324c2444"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:45:55.303166 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.303107 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r"] Sep 06 10:45:55.303520 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.303497 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e466c20d-453b-4c70-94f2-3265324c2444-maas-api-service-ca\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:45:55.304213 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:45:55.304190 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce904edc_a4bc_4738_aa3c_80de703e9790.slice/crio-bf5cda2e5ff6174891998a699338a85d372595bf95740f5ff9bfe9c7f0fac5ba WatchSource:0}: Error finding container bf5cda2e5ff6174891998a699338a85d372595bf95740f5ff9bfe9c7f0fac5ba: Status 404 returned error can't find the container with id bf5cda2e5ff6174891998a699338a85d372595bf95740f5ff9bfe9c7f0fac5ba Sep 06 10:45:55.982433 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.982404 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh" Sep 06 10:45:55.982619 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.982405 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh" event={"ID":"e466c20d-453b-4c70-94f2-3265324c2444","Type":"ContainerDied","Data":"744c69bff06abccaaa1a3868ee575676c6bd5775bfc95a4d21c6e170f40faacc"} Sep 06 10:45:55.982619 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.982532 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="744c69bff06abccaaa1a3868ee575676c6bd5775bfc95a4d21c6e170f40faacc" Sep 06 10:45:55.984244 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.984208 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" event={"ID":"ce904edc-a4bc-4738-aa3c-80de703e9790","Type":"ContainerStarted","Data":"a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f"} Sep 06 10:45:55.984364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.984254 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" event={"ID":"ce904edc-a4bc-4738-aa3c-80de703e9790","Type":"ContainerStarted","Data":"bf5cda2e5ff6174891998a699338a85d372595bf95740f5ff9bfe9c7f0fac5ba"} Sep 06 10:45:55.984364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.984284 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" Sep 06 10:45:55.999360 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:45:55.999321 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" podStartSLOduration=1.9993099540000001 podStartE2EDuration="1.999309954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:45:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:45:55.998137024 +0000 UTC m=+949.580626647" watchObservedRunningTime="2026-09-06 10:45:55.999309954 +0000 UTC m=+949.581799584" Sep 06 10:46:02.137971 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:02.137932 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:02.690462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:02.690432 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:02.711311 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:02.711277 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb"] Sep 06 10:46:02.711557 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:02.711530 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" podUID="9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c" containerName="payload-processing" containerID="cri-o://c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964" gracePeriod=30 Sep 06 10:46:02.791173 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:02.791145 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:02.861901 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:02.861863 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:02.862936 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:02.862913 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh"] Sep 06 10:46:02.865117 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:02.865089 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2313d276-5mhnh"] Sep 06 10:46:02.917146 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:02.917059 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:02.928195 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:02.928114 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e466c20d-453b-4c70-94f2-3265324c2444" path="/var/lib/kubelet/pods/e466c20d-453b-4c70-94f2-3265324c2444/volumes" Sep 06 10:46:03.011597 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.011575 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" Sep 06 10:46:03.024682 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.024652 2574 generic.go:356] "Generic (PLEG): container finished" podID="9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c" containerID="c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964" exitCode=0 Sep 06 10:46:03.024812 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.024697 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" Sep 06 10:46:03.024812 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.024699 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" event={"ID":"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c","Type":"ContainerDied","Data":"c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964"} Sep 06 10:46:03.024812 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.024755 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb" event={"ID":"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c","Type":"ContainerDied","Data":"e0a3151639d0a8878b40ef1b7835896ddcbd625d987eaa3b7571d640b8295a5a"} Sep 06 10:46:03.024812 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.024777 2574 scope.go:117] "RemoveContainer" containerID="c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964" Sep 06 10:46:03.061248 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.061224 2574 scope.go:117] "RemoveContainer" containerID="c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964" Sep 06 10:46:03.062140 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:46:03.062095 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964\": container with ID starting with c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964 not found: ID does not exist" containerID="c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964" Sep 06 10:46:03.062278 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.062159 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964"} err="failed to get container status \"c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964\": rpc error: code = NotFound desc = could not find container \"c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964\": container with ID starting with c1a21745238ac9f98664874dcdf9ffa1016fdf740ef69d0c708c6d293f76a964 not found: ID does not exist" Sep 06 10:46:03.079800 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.079778 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fg9tc\" (UniqueName: \"kubernetes.io/projected/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c-kube-api-access-fg9tc\") pod \"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c\" (UID: \"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c\") " Sep 06 10:46:03.079914 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.079848 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c-plugins-config-volume\") pod \"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c\" (UID: \"9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c\") " Sep 06 10:46:03.080203 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.080179 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c" (UID: "9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:46:03.081626 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.081607 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c-kube-api-access-fg9tc" (OuterVolumeSpecName: "kube-api-access-fg9tc") pod "9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c" (UID: "9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c"). InnerVolumeSpecName "kube-api-access-fg9tc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:46:03.180550 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.180519 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:46:03.180550 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.180546 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg9tc\" (UniqueName: \"kubernetes.io/projected/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c-kube-api-access-fg9tc\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:46:03.215582 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.215537 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:03.350221 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.350165 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb"] Sep 06 10:46:03.357403 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:03.357367 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2313d276-868b8bbc45-hf2hb"] Sep 06 10:46:04.927239 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:04.927195 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c" path="/var/lib/kubelet/pods/9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c/volumes" Sep 06 10:46:06.992386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:06.992355 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" Sep 06 10:46:07.998350 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:07.998301 2574 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 06 10:46:07.998809 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:07.998388 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 06 10:46:08.787404 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:08.787370 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:08.849898 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:08.849869 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:09.040081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:09.039992 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:09.086240 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:09.086209 2574 generic.go:356] "Generic (PLEG): container finished" podID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerID="e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e" exitCode=0 Sep 06 10:46:09.086410 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:09.086255 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" event={"ID":"045c7a18-e0f4-4e90-945c-d6512aa7d1f4","Type":"ContainerDied","Data":"e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e"} Sep 06 10:46:09.086705 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:09.086690 2574 scope.go:117] "RemoveContainer" containerID="e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e" Sep 06 10:46:09.087794 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:09.087765 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:46:09.087846 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:09.087832 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:46:09.087887 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:09.087863 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 06 10:46:09.382940 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:09.382853 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:09.489400 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:09.489362 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:09.606425 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:09.606392 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:10.093438 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:10.093400 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" event={"ID":"045c7a18-e0f4-4e90-945c-d6512aa7d1f4","Type":"ContainerStarted","Data":"a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384"} Sep 06 10:46:10.159999 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:10.159948 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:46:10.160222 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:10.160196 2574 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Sep 06 10:46:10.160283 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:10.160248 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Sep 06 10:46:11.159766 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:11.159712 2574 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Sep 06 10:46:11.160272 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:11.159782 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Sep 06 10:46:12.159816 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:12.159779 2574 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Sep 06 10:46:12.160268 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:12.159835 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Sep 06 10:46:13.159889 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:13.159848 2574 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Sep 06 10:46:13.160321 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:13.159914 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Sep 06 10:46:14.159852 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:14.159814 2574 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Sep 06 10:46:14.160033 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:14.159875 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Sep 06 10:46:15.307296 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:15.307264 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:15.352695 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:15.352660 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:46:15.353070 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:15.353033 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:46:15.353910 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:15.353891 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:46:15.366246 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:15.366223 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:15.466593 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:15.466551 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:15.572756 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:15.572672 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:15.702767 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:15.702732 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:15.823061 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:15.822960 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:17.793570 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:17.793524 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:18.120680 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:18.120583 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:18.406964 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:18.406871 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:18.842604 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:18.842550 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:21.212800 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:21.212757 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:21.310668 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:21.310638 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:21.410770 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:21.410737 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:22.220087 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:22.220052 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:22.433069 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:22.433034 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:22.514631 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:22.514558 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:22.583140 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:22.583077 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:22.697056 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:22.697015 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:22.913543 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:22.913462 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:23.013398 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.013360 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:23.015406 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.015373 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2"] Sep 06 10:46:23.015785 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.015768 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c" containerName="payload-processing" Sep 06 10:46:23.015785 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.015785 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c" containerName="payload-processing" Sep 06 10:46:23.015942 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.015813 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e466c20d-453b-4c70-94f2-3265324c2444" containerName="revoke-keys" Sep 06 10:46:23.015942 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.015824 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e466c20d-453b-4c70-94f2-3265324c2444" containerName="revoke-keys" Sep 06 10:46:23.015942 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.015892 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b3b7a51-d0f1-46f7-b74c-c1f6601a9e0c" containerName="payload-processing" Sep 06 10:46:23.015942 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.015904 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e466c20d-453b-4c70-94f2-3265324c2444" containerName="revoke-keys" Sep 06 10:46:23.019427 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.019408 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" Sep 06 10:46:23.021552 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.021530 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-eb4b45\"" Sep 06 10:46:23.021782 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.021765 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-eb4b45-dockercfg-ls79f\"" Sep 06 10:46:23.025290 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.025266 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv"] Sep 06 10:46:23.031272 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.031250 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2"] Sep 06 10:46:23.031385 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.031333 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" Sep 06 10:46:23.043519 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.043496 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv"] Sep 06 10:46:23.072337 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.072306 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/090b8b25-589c-41d0-8bda-6df732167703-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2\" (UID: \"090b8b25-589c-41d0-8bda-6df732167703\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" Sep 06 10:46:23.072455 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.072366 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gqg8\" (UniqueName: \"kubernetes.io/projected/090b8b25-589c-41d0-8bda-6df732167703-kube-api-access-9gqg8\") pod \"payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2\" (UID: \"090b8b25-589c-41d0-8bda-6df732167703\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" Sep 06 10:46:23.072512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.072481 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsmc9\" (UniqueName: \"kubernetes.io/projected/48b48797-0260-4391-9fe6-ab7eaa3b73c7-kube-api-access-gsmc9\") pod \"payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv\" (UID: \"48b48797-0260-4391-9fe6-ab7eaa3b73c7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" Sep 06 10:46:23.072562 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.072527 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48b48797-0260-4391-9fe6-ab7eaa3b73c7-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv\" (UID: \"48b48797-0260-4391-9fe6-ab7eaa3b73c7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" Sep 06 10:46:23.173065 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.173036 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsmc9\" (UniqueName: \"kubernetes.io/projected/48b48797-0260-4391-9fe6-ab7eaa3b73c7-kube-api-access-gsmc9\") pod \"payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv\" (UID: \"48b48797-0260-4391-9fe6-ab7eaa3b73c7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" Sep 06 10:46:23.173278 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.173079 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48b48797-0260-4391-9fe6-ab7eaa3b73c7-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv\" (UID: \"48b48797-0260-4391-9fe6-ab7eaa3b73c7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" Sep 06 10:46:23.173278 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.173117 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/090b8b25-589c-41d0-8bda-6df732167703-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2\" (UID: \"090b8b25-589c-41d0-8bda-6df732167703\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" Sep 06 10:46:23.173278 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.173173 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gqg8\" (UniqueName: \"kubernetes.io/projected/090b8b25-589c-41d0-8bda-6df732167703-kube-api-access-9gqg8\") pod \"payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2\" (UID: \"090b8b25-589c-41d0-8bda-6df732167703\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" Sep 06 10:46:23.174054 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.174026 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/090b8b25-589c-41d0-8bda-6df732167703-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2\" (UID: \"090b8b25-589c-41d0-8bda-6df732167703\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" Sep 06 10:46:23.174187 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.174171 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48b48797-0260-4391-9fe6-ab7eaa3b73c7-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv\" (UID: \"48b48797-0260-4391-9fe6-ab7eaa3b73c7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" Sep 06 10:46:23.180584 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.180557 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsmc9\" (UniqueName: \"kubernetes.io/projected/48b48797-0260-4391-9fe6-ab7eaa3b73c7-kube-api-access-gsmc9\") pod \"payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv\" (UID: \"48b48797-0260-4391-9fe6-ab7eaa3b73c7\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" Sep 06 10:46:23.180691 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.180671 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gqg8\" (UniqueName: \"kubernetes.io/projected/090b8b25-589c-41d0-8bda-6df732167703-kube-api-access-9gqg8\") pod \"payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2\" (UID: \"090b8b25-589c-41d0-8bda-6df732167703\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" Sep 06 10:46:23.332603 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.332572 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" Sep 06 10:46:23.341710 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.341685 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" Sep 06 10:46:23.426417 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.426322 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:23.615382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.615355 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2"] Sep 06 10:46:23.616632 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:46:23.616602 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod090b8b25_589c_41d0_8bda_6df732167703.slice/crio-d150f036e519d26e42a13b33bc52e0f3e89a4c1c201d2a03c77c57a8b85e088b WatchSource:0}: Error finding container d150f036e519d26e42a13b33bc52e0f3e89a4c1c201d2a03c77c57a8b85e088b: Status 404 returned error can't find the container with id d150f036e519d26e42a13b33bc52e0f3e89a4c1c201d2a03c77c57a8b85e088b Sep 06 10:46:23.629340 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:23.629320 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv"] Sep 06 10:46:23.630933 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:46:23.630910 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b48797_0260_4391_9fe6_ab7eaa3b73c7.slice/crio-c8f63a18e46dc83d35220f87526f35ce50e6b84e5c30c6dfc7e336917a8918c0 WatchSource:0}: Error finding container c8f63a18e46dc83d35220f87526f35ce50e6b84e5c30c6dfc7e336917a8918c0: Status 404 returned error can't find the container with id c8f63a18e46dc83d35220f87526f35ce50e6b84e5c30c6dfc7e336917a8918c0 Sep 06 10:46:24.170496 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:24.170460 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" event={"ID":"48b48797-0260-4391-9fe6-ab7eaa3b73c7","Type":"ContainerStarted","Data":"e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c"} Sep 06 10:46:24.170680 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:24.170516 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" event={"ID":"48b48797-0260-4391-9fe6-ab7eaa3b73c7","Type":"ContainerStarted","Data":"c8f63a18e46dc83d35220f87526f35ce50e6b84e5c30c6dfc7e336917a8918c0"} Sep 06 10:46:24.170680 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:24.170550 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" Sep 06 10:46:24.172212 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:24.172182 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" event={"ID":"090b8b25-589c-41d0-8bda-6df732167703","Type":"ContainerStarted","Data":"7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308"} Sep 06 10:46:24.172301 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:24.172234 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" event={"ID":"090b8b25-589c-41d0-8bda-6df732167703","Type":"ContainerStarted","Data":"d150f036e519d26e42a13b33bc52e0f3e89a4c1c201d2a03c77c57a8b85e088b"} Sep 06 10:46:24.172301 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:24.172256 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" Sep 06 10:46:24.188697 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:24.187871 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" podStartSLOduration=1.187854212 podStartE2EDuration="1.187854212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:46:24.184633031 +0000 UTC m=+977.767122665" watchObservedRunningTime="2026-09-06 10:46:24.187854212 +0000 UTC m=+977.770343845" Sep 06 10:46:24.201193 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:24.201150 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" podStartSLOduration=2.201134497 podStartE2EDuration="2.201134497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:46:24.198539112 +0000 UTC m=+977.781028748" watchObservedRunningTime="2026-09-06 10:46:24.201134497 +0000 UTC m=+977.783624118" Sep 06 10:46:25.265075 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:25.265036 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:25.358297 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:25.358260 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:26.639965 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:26.639934 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:26.707279 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:26.707241 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:26.801517 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:26.801480 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:27.101582 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.101552 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:27.190672 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.190636 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:27.230062 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.230028 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725"] Sep 06 10:46:27.233752 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.233735 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" Sep 06 10:46:27.236030 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.235996 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-72cb16\"" Sep 06 10:46:27.236165 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.236055 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-72cb16-dockercfg-crw9f\"" Sep 06 10:46:27.244777 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.244750 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725"] Sep 06 10:46:27.309579 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.309549 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vbsz\" (UniqueName: \"kubernetes.io/projected/2fcd3188-40ca-4863-b0cb-12780020c1f4-kube-api-access-2vbsz\") pod \"payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725\" (UID: \"2fcd3188-40ca-4863-b0cb-12780020c1f4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" Sep 06 10:46:27.309736 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.309626 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fcd3188-40ca-4863-b0cb-12780020c1f4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725\" (UID: \"2fcd3188-40ca-4863-b0cb-12780020c1f4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" Sep 06 10:46:27.411027 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.410944 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vbsz\" (UniqueName: \"kubernetes.io/projected/2fcd3188-40ca-4863-b0cb-12780020c1f4-kube-api-access-2vbsz\") pod \"payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725\" (UID: \"2fcd3188-40ca-4863-b0cb-12780020c1f4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" Sep 06 10:46:27.411027 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.411021 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fcd3188-40ca-4863-b0cb-12780020c1f4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725\" (UID: \"2fcd3188-40ca-4863-b0cb-12780020c1f4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" Sep 06 10:46:27.411730 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.411710 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fcd3188-40ca-4863-b0cb-12780020c1f4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725\" (UID: \"2fcd3188-40ca-4863-b0cb-12780020c1f4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" Sep 06 10:46:27.419039 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.419020 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vbsz\" (UniqueName: \"kubernetes.io/projected/2fcd3188-40ca-4863-b0cb-12780020c1f4-kube-api-access-2vbsz\") pod \"payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725\" (UID: \"2fcd3188-40ca-4863-b0cb-12780020c1f4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" Sep 06 10:46:27.545438 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.545398 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" Sep 06 10:46:27.814761 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:27.814731 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725"] Sep 06 10:46:27.816364 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:46:27.816337 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fcd3188_40ca_4863_b0cb_12780020c1f4.slice/crio-43788f4816b1b7b426d3efba57d2ee38a60ce81b92a03acebf1db154c5a44851 WatchSource:0}: Error finding container 43788f4816b1b7b426d3efba57d2ee38a60ce81b92a03acebf1db154c5a44851: Status 404 returned error can't find the container with id 43788f4816b1b7b426d3efba57d2ee38a60ce81b92a03acebf1db154c5a44851 Sep 06 10:46:28.195599 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:28.195563 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" event={"ID":"2fcd3188-40ca-4863-b0cb-12780020c1f4","Type":"ContainerStarted","Data":"7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3"} Sep 06 10:46:28.195790 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:28.195630 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" Sep 06 10:46:28.195790 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:28.195654 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" event={"ID":"2fcd3188-40ca-4863-b0cb-12780020c1f4","Type":"ContainerStarted","Data":"43788f4816b1b7b426d3efba57d2ee38a60ce81b92a03acebf1db154c5a44851"} Sep 06 10:46:28.211334 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:28.211286 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" podStartSLOduration=1.21127214 podStartE2EDuration="1.21127214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:46:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:46:28.209878487 +0000 UTC m=+981.792368119" watchObservedRunningTime="2026-09-06 10:46:28.21127214 +0000 UTC m=+981.793761769" Sep 06 10:46:35.179902 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:35.179825 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" Sep 06 10:46:35.179902 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:35.179889 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" Sep 06 10:46:39.203615 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:39.203583 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" Sep 06 10:46:43.276993 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:43.276954 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:46.165812 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:46.165770 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:47.616180 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:47.616140 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:50.219943 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:50.219890 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:50.545714 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:50.545602 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:50.852962 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:50.852865 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:51.316709 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:51.316675 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:54.423241 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:54.423183 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:54.521748 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:54.521715 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:55.568394 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:55.568357 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:55.675845 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:55.675806 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:55.977713 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:55.977678 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:56.081874 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.081836 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:56.157394 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.157359 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:46:56.162403 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.162378 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94"] Sep 06 10:46:56.165999 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.165984 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" Sep 06 10:46:56.168697 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.168679 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-fbf820-dockercfg-xfmqg\"" Sep 06 10:46:56.168981 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.168964 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-fbf820\"" Sep 06 10:46:56.179674 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.179650 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94"] Sep 06 10:46:56.193304 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.193277 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2"] Sep 06 10:46:56.196825 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.196808 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" Sep 06 10:46:56.205948 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.205926 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2"] Sep 06 10:46:56.287439 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.287330 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv6ss\" (UniqueName: \"kubernetes.io/projected/5e4edc92-6408-406d-a71a-80f096fa426a-kube-api-access-rv6ss\") pod \"payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94\" (UID: \"5e4edc92-6408-406d-a71a-80f096fa426a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" Sep 06 10:46:56.287613 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.287442 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e4edc92-6408-406d-a71a-80f096fa426a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94\" (UID: \"5e4edc92-6408-406d-a71a-80f096fa426a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" Sep 06 10:46:56.287613 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.287526 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975eb1da-1333-42fb-ad66-7cf320a65fd1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2\" (UID: \"975eb1da-1333-42fb-ad66-7cf320a65fd1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" Sep 06 10:46:56.287714 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.287610 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc74s\" (UniqueName: \"kubernetes.io/projected/975eb1da-1333-42fb-ad66-7cf320a65fd1-kube-api-access-vc74s\") pod \"payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2\" (UID: \"975eb1da-1333-42fb-ad66-7cf320a65fd1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" Sep 06 10:46:56.388901 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.388867 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc74s\" (UniqueName: \"kubernetes.io/projected/975eb1da-1333-42fb-ad66-7cf320a65fd1-kube-api-access-vc74s\") pod \"payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2\" (UID: \"975eb1da-1333-42fb-ad66-7cf320a65fd1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" Sep 06 10:46:56.389059 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.388936 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv6ss\" (UniqueName: \"kubernetes.io/projected/5e4edc92-6408-406d-a71a-80f096fa426a-kube-api-access-rv6ss\") pod \"payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94\" (UID: \"5e4edc92-6408-406d-a71a-80f096fa426a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" Sep 06 10:46:56.389059 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.388960 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e4edc92-6408-406d-a71a-80f096fa426a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94\" (UID: \"5e4edc92-6408-406d-a71a-80f096fa426a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" Sep 06 10:46:56.389059 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.388990 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975eb1da-1333-42fb-ad66-7cf320a65fd1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2\" (UID: \"975eb1da-1333-42fb-ad66-7cf320a65fd1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" Sep 06 10:46:56.389681 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.389658 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975eb1da-1333-42fb-ad66-7cf320a65fd1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2\" (UID: \"975eb1da-1333-42fb-ad66-7cf320a65fd1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" Sep 06 10:46:56.389773 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.389717 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e4edc92-6408-406d-a71a-80f096fa426a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94\" (UID: \"5e4edc92-6408-406d-a71a-80f096fa426a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" Sep 06 10:46:56.397881 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.397743 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv6ss\" (UniqueName: \"kubernetes.io/projected/5e4edc92-6408-406d-a71a-80f096fa426a-kube-api-access-rv6ss\") pod \"payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94\" (UID: \"5e4edc92-6408-406d-a71a-80f096fa426a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" Sep 06 10:46:56.397881 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.397806 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc74s\" (UniqueName: \"kubernetes.io/projected/975eb1da-1333-42fb-ad66-7cf320a65fd1-kube-api-access-vc74s\") pod \"payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2\" (UID: \"975eb1da-1333-42fb-ad66-7cf320a65fd1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" Sep 06 10:46:56.475948 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.475909 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" Sep 06 10:46:56.508879 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.508623 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" Sep 06 10:46:56.750825 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.750798 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94"] Sep 06 10:46:56.752282 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:46:56.752258 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e4edc92_6408_406d_a71a_80f096fa426a.slice/crio-9e86026326aa181024b87a5449d2cf5786e0cf94f67a0699fbb77c041576301e WatchSource:0}: Error finding container 9e86026326aa181024b87a5449d2cf5786e0cf94f67a0699fbb77c041576301e: Status 404 returned error can't find the container with id 9e86026326aa181024b87a5449d2cf5786e0cf94f67a0699fbb77c041576301e Sep 06 10:46:56.792689 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:56.792661 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2"] Sep 06 10:46:56.793451 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:46:56.793418 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod975eb1da_1333_42fb_ad66_7cf320a65fd1.slice/crio-7ff0e9c25bfb82905c6ba2b31e28e16693d04d70f9c8d4e28d85c201471bdf7d WatchSource:0}: Error finding container 7ff0e9c25bfb82905c6ba2b31e28e16693d04d70f9c8d4e28d85c201471bdf7d: Status 404 returned error can't find the container with id 7ff0e9c25bfb82905c6ba2b31e28e16693d04d70f9c8d4e28d85c201471bdf7d Sep 06 10:46:57.365953 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:57.365910 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" event={"ID":"975eb1da-1333-42fb-ad66-7cf320a65fd1","Type":"ContainerStarted","Data":"63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585"} Sep 06 10:46:57.366174 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:57.365979 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" event={"ID":"975eb1da-1333-42fb-ad66-7cf320a65fd1","Type":"ContainerStarted","Data":"7ff0e9c25bfb82905c6ba2b31e28e16693d04d70f9c8d4e28d85c201471bdf7d"} Sep 06 10:46:57.366174 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:57.366060 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" Sep 06 10:46:57.367657 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:57.367634 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" event={"ID":"5e4edc92-6408-406d-a71a-80f096fa426a","Type":"ContainerStarted","Data":"534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec"} Sep 06 10:46:57.367657 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:57.367669 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" event={"ID":"5e4edc92-6408-406d-a71a-80f096fa426a","Type":"ContainerStarted","Data":"9e86026326aa181024b87a5449d2cf5786e0cf94f67a0699fbb77c041576301e"} Sep 06 10:46:57.367842 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:57.367767 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" Sep 06 10:46:57.382632 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:57.382583 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" podStartSLOduration=1.382567444 podStartE2EDuration="1.382567444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:46:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:46:57.380534984 +0000 UTC m=+1010.963024628" watchObservedRunningTime="2026-09-06 10:46:57.382567444 +0000 UTC m=+1010.965057075" Sep 06 10:46:57.395188 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:46:57.395141 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" podStartSLOduration=1.395110278 podStartE2EDuration="1.395110278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:46:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:46:57.394871904 +0000 UTC m=+1010.977361535" watchObservedRunningTime="2026-09-06 10:46:57.395110278 +0000 UTC m=+1010.977599913" Sep 06 10:47:05.558051 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:05.557991 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:08.377305 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:08.377270 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" Sep 06 10:47:08.377773 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:08.377349 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" Sep 06 10:47:19.354895 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:19.354846 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:32.578224 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:32.578189 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:32.868482 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:32.868386 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:33.160483 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:33.160391 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:35.696216 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:35.696174 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:35.789840 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:35.789808 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:36.946091 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:36.946056 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:37.007965 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.007932 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:37.300061 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.299978 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:37.405459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.405424 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:37.515087 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.515054 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:37.551087 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.550996 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk"] Sep 06 10:47:37.554509 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.554487 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" Sep 06 10:47:37.556702 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.556682 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-aec3c2bf-dockercfg-8fg8q\"" Sep 06 10:47:37.556803 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.556728 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-aec3c2bf\"" Sep 06 10:47:37.564826 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.564805 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk"] Sep 06 10:47:37.679933 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.679897 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqd6f\" (UniqueName: \"kubernetes.io/projected/fa5d66a5-9401-4634-ab85-dee8ed854447-kube-api-access-cqd6f\") pod \"payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk\" (UID: \"fa5d66a5-9401-4634-ab85-dee8ed854447\") " pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" Sep 06 10:47:37.680115 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.679958 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5d66a5-9401-4634-ab85-dee8ed854447-plugins-config-volume\") pod \"payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk\" (UID: \"fa5d66a5-9401-4634-ab85-dee8ed854447\") " pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" Sep 06 10:47:37.781524 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.781488 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5d66a5-9401-4634-ab85-dee8ed854447-plugins-config-volume\") pod \"payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk\" (UID: \"fa5d66a5-9401-4634-ab85-dee8ed854447\") " pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" Sep 06 10:47:37.781685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.781617 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqd6f\" (UniqueName: \"kubernetes.io/projected/fa5d66a5-9401-4634-ab85-dee8ed854447-kube-api-access-cqd6f\") pod \"payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk\" (UID: \"fa5d66a5-9401-4634-ab85-dee8ed854447\") " pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" Sep 06 10:47:37.782307 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.782287 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5d66a5-9401-4634-ab85-dee8ed854447-plugins-config-volume\") pod \"payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk\" (UID: \"fa5d66a5-9401-4634-ab85-dee8ed854447\") " pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" Sep 06 10:47:37.789628 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.789601 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqd6f\" (UniqueName: \"kubernetes.io/projected/fa5d66a5-9401-4634-ab85-dee8ed854447-kube-api-access-cqd6f\") pod \"payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk\" (UID: \"fa5d66a5-9401-4634-ab85-dee8ed854447\") " pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" Sep 06 10:47:37.866676 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:37.866585 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" Sep 06 10:47:38.141446 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:38.141419 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk"] Sep 06 10:47:38.142719 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:47:38.142693 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa5d66a5_9401_4634_ab85_dee8ed854447.slice/crio-79053b5579712bb85d37ea2017912afe8bfabf950928b6fa28dea2a18a27fee6 WatchSource:0}: Error finding container 79053b5579712bb85d37ea2017912afe8bfabf950928b6fa28dea2a18a27fee6: Status 404 returned error can't find the container with id 79053b5579712bb85d37ea2017912afe8bfabf950928b6fa28dea2a18a27fee6 Sep 06 10:47:38.605151 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:38.605102 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" event={"ID":"fa5d66a5-9401-4634-ab85-dee8ed854447","Type":"ContainerStarted","Data":"dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90"} Sep 06 10:47:38.605336 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:38.605166 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" event={"ID":"fa5d66a5-9401-4634-ab85-dee8ed854447","Type":"ContainerStarted","Data":"79053b5579712bb85d37ea2017912afe8bfabf950928b6fa28dea2a18a27fee6"} Sep 06 10:47:38.605336 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:38.605324 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" Sep 06 10:47:38.621938 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:38.621892 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" podStartSLOduration=1.621879335 podStartE2EDuration="1.621879335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:47:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:47:38.619481205 +0000 UTC m=+1052.201970834" watchObservedRunningTime="2026-09-06 10:47:38.621879335 +0000 UTC m=+1052.204368966" Sep 06 10:47:49.613381 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:49.613351 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" Sep 06 10:47:56.510727 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:56.510693 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:47:56.567772 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:47:56.567733 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:48:00.654344 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:00.654260 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:48:02.854866 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:02.854821 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:48:14.229474 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:14.229439 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:48:18.477337 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:18.477300 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:48:21.423877 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.423843 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz"] Sep 06 10:48:21.427444 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.427416 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.430541 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.430520 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-aec3c2bf\"/\"e2e-lifecycle-model-aec3c2bf-kserve-self-signed-certs\"" Sep 06 10:48:21.430665 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.430520 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-aec3c2bf\"/\"kube-root-ca.crt\"" Sep 06 10:48:21.430665 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.430568 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-aec3c2bf\"/\"openshift-service-ca.crt\"" Sep 06 10:48:21.430795 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.430574 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-aec3c2bf\"/\"default-dockercfg-tjqtf\"" Sep 06 10:48:21.435925 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.435901 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz"] Sep 06 10:48:21.463688 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.463656 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:48:21.594766 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.594730 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/498da61b-8c07-44ae-9847-df1791622a74-tls-certs\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.594989 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.594772 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-model-cache\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.594989 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.594796 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-dshm\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.594989 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.594904 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-home\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.594989 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.594956 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqw76\" (UniqueName: \"kubernetes.io/projected/498da61b-8c07-44ae-9847-df1791622a74-kube-api-access-zqw76\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.595195 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.595020 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-tmp-dir\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.696189 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.696071 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-tmp-dir\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.696189 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.696162 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/498da61b-8c07-44ae-9847-df1791622a74-tls-certs\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.696189 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.696185 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-model-cache\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.696479 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.696205 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-dshm\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.696479 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.696280 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-home\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.696479 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.696332 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqw76\" (UniqueName: \"kubernetes.io/projected/498da61b-8c07-44ae-9847-df1791622a74-kube-api-access-zqw76\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.696630 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.696540 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-tmp-dir\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.696630 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.696584 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-model-cache\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.696630 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.696623 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-home\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.698484 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.698463 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-dshm\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.698627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.698612 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/498da61b-8c07-44ae-9847-df1791622a74-tls-certs\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.703416 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.703398 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqw76\" (UniqueName: \"kubernetes.io/projected/498da61b-8c07-44ae-9847-df1791622a74-kube-api-access-zqw76\") pod \"e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:21.738625 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:21.738581 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:22.010077 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:22.010050 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz"] Sep 06 10:48:22.011430 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:48:22.011391 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod498da61b_8c07_44ae_9847_df1791622a74.slice/crio-545ab347c34ca265907ac7cbe1b3c77c3670b96fbe70f6f94d5cce9edcd7fb74 WatchSource:0}: Error finding container 545ab347c34ca265907ac7cbe1b3c77c3670b96fbe70f6f94d5cce9edcd7fb74: Status 404 returned error can't find the container with id 545ab347c34ca265907ac7cbe1b3c77c3670b96fbe70f6f94d5cce9edcd7fb74 Sep 06 10:48:22.858610 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:22.858572 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" event={"ID":"498da61b-8c07-44ae-9847-df1791622a74","Type":"ContainerStarted","Data":"292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b"} Sep 06 10:48:22.858990 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:22.858624 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" event={"ID":"498da61b-8c07-44ae-9847-df1791622a74","Type":"ContainerStarted","Data":"545ab347c34ca265907ac7cbe1b3c77c3670b96fbe70f6f94d5cce9edcd7fb74"} Sep 06 10:48:22.874348 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:22.874302 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" podStartSLOduration=1.874287087 podStartE2EDuration="1.874287087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:48:22.87309681 +0000 UTC m=+1096.455586442" watchObservedRunningTime="2026-09-06 10:48:22.874287087 +0000 UTC m=+1096.456776717" Sep 06 10:48:31.739022 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:31.738978 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:31.739022 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:31.739027 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:31.751657 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:31.751634 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:31.920528 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:31.920498 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:48:44.054296 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:44.054255 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:48:55.454911 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:55.454872 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk"] Sep 06 10:48:55.458686 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:55.458668 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk" Sep 06 10:48:55.461050 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:55.461022 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\"" Sep 06 10:48:55.461351 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:55.461200 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 10:48:55.465901 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:55.465877 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk"] Sep 06 10:48:55.516814 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:55.516771 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed489e30-b06a-4123-8c7a-6eda7e1f23eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk\" (UID: \"ed489e30-b06a-4123-8c7a-6eda7e1f23eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk" Sep 06 10:48:55.617986 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:55.617941 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed489e30-b06a-4123-8c7a-6eda7e1f23eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk\" (UID: \"ed489e30-b06a-4123-8c7a-6eda7e1f23eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk" Sep 06 10:48:55.618685 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:55.618647 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed489e30-b06a-4123-8c7a-6eda7e1f23eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk\" (UID: \"ed489e30-b06a-4123-8c7a-6eda7e1f23eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk" Sep 06 10:48:55.669976 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:55.669933 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:48:55.772437 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:55.772336 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk" Sep 06 10:48:56.054818 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:56.054779 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk"] Sep 06 10:48:57.067577 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:57.067492 2574 generic.go:356] "Generic (PLEG): container finished" podID="ed489e30-b06a-4123-8c7a-6eda7e1f23eb" containerID="289d632e8a420bee3e7c23de447da2b7c6881ca12499745f68fba5515f7e4171" exitCode=0 Sep 06 10:48:57.067577 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:57.067530 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk" event={"ID":"ed489e30-b06a-4123-8c7a-6eda7e1f23eb","Type":"ContainerDied","Data":"289d632e8a420bee3e7c23de447da2b7c6881ca12499745f68fba5515f7e4171"} Sep 06 10:48:57.067577 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:57.067567 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk" event={"ID":"ed489e30-b06a-4123-8c7a-6eda7e1f23eb","Type":"ContainerStarted","Data":"d453789b176e121dc4055f0377f866daa5a726f4205449d291e1885f708c9108"} Sep 06 10:48:58.207686 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:58.207663 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk" Sep 06 10:48:58.344784 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:58.344698 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed489e30-b06a-4123-8c7a-6eda7e1f23eb-maas-api-service-ca\") pod \"ed489e30-b06a-4123-8c7a-6eda7e1f23eb\" (UID: \"ed489e30-b06a-4123-8c7a-6eda7e1f23eb\") " Sep 06 10:48:58.345040 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:58.345017 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed489e30-b06a-4123-8c7a-6eda7e1f23eb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ed489e30-b06a-4123-8c7a-6eda7e1f23eb" (UID: "ed489e30-b06a-4123-8c7a-6eda7e1f23eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:48:58.446101 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:58.446068 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed489e30-b06a-4123-8c7a-6eda7e1f23eb-maas-api-service-ca\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:48:59.079584 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:59.079553 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk" Sep 06 10:48:59.079584 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:59.079568 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk" event={"ID":"ed489e30-b06a-4123-8c7a-6eda7e1f23eb","Type":"ContainerDied","Data":"d453789b176e121dc4055f0377f866daa5a726f4205449d291e1885f708c9108"} Sep 06 10:48:59.079788 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:48:59.079620 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d453789b176e121dc4055f0377f866daa5a726f4205449d291e1885f708c9108" Sep 06 10:49:05.106007 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:05.105969 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:49:11.169895 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.169862 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:49:11.198034 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.198002 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk"] Sep 06 10:49:11.198289 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.198262 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" podUID="fa5d66a5-9401-4634-ab85-dee8ed854447" containerName="payload-processing" containerID="cri-o://dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90" gracePeriod=30 Sep 06 10:49:11.240504 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.240470 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:49:11.300257 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.300224 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:49:11.381504 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.381450 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk"] Sep 06 10:49:11.382784 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.382756 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:49:11.384267 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.384240 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-aec3c2bf-wxffk"] Sep 06 10:49:11.440149 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.440015 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:49:11.462322 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.462298 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" Sep 06 10:49:11.574622 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.574589 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5d66a5-9401-4634-ab85-dee8ed854447-plugins-config-volume\") pod \"fa5d66a5-9401-4634-ab85-dee8ed854447\" (UID: \"fa5d66a5-9401-4634-ab85-dee8ed854447\") " Sep 06 10:49:11.574815 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.574663 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqd6f\" (UniqueName: \"kubernetes.io/projected/fa5d66a5-9401-4634-ab85-dee8ed854447-kube-api-access-cqd6f\") pod \"fa5d66a5-9401-4634-ab85-dee8ed854447\" (UID: \"fa5d66a5-9401-4634-ab85-dee8ed854447\") " Sep 06 10:49:11.574994 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.574969 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa5d66a5-9401-4634-ab85-dee8ed854447-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fa5d66a5-9401-4634-ab85-dee8ed854447" (UID: "fa5d66a5-9401-4634-ab85-dee8ed854447"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:49:11.576730 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.576704 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa5d66a5-9401-4634-ab85-dee8ed854447-kube-api-access-cqd6f" (OuterVolumeSpecName: "kube-api-access-cqd6f") pod "fa5d66a5-9401-4634-ab85-dee8ed854447" (UID: "fa5d66a5-9401-4634-ab85-dee8ed854447"). InnerVolumeSpecName "kube-api-access-cqd6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:49:11.676152 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.676087 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqd6f\" (UniqueName: \"kubernetes.io/projected/fa5d66a5-9401-4634-ab85-dee8ed854447-kube-api-access-cqd6f\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:49:11.676152 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:11.676150 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa5d66a5-9401-4634-ab85-dee8ed854447-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:49:12.168599 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:12.168560 2574 generic.go:356] "Generic (PLEG): container finished" podID="fa5d66a5-9401-4634-ab85-dee8ed854447" containerID="dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90" exitCode=0 Sep 06 10:49:12.168801 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:12.168607 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" Sep 06 10:49:12.168801 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:12.168650 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" event={"ID":"fa5d66a5-9401-4634-ab85-dee8ed854447","Type":"ContainerDied","Data":"dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90"} Sep 06 10:49:12.168801 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:12.168705 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk" event={"ID":"fa5d66a5-9401-4634-ab85-dee8ed854447","Type":"ContainerDied","Data":"79053b5579712bb85d37ea2017912afe8bfabf950928b6fa28dea2a18a27fee6"} Sep 06 10:49:12.168801 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:12.168733 2574 scope.go:117] "RemoveContainer" containerID="dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90" Sep 06 10:49:12.178398 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:12.178381 2574 scope.go:117] "RemoveContainer" containerID="dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90" Sep 06 10:49:12.178666 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:49:12.178640 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90\": container with ID starting with dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90 not found: ID does not exist" containerID="dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90" Sep 06 10:49:12.178707 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:12.178663 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90"} err="failed to get container status \"dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90\": rpc error: code = NotFound desc = could not find container \"dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90\": container with ID starting with dc339687dc5253893938739312f466ae9308d63ab5e9a011e6c3ecba1f35ba90 not found: ID does not exist" Sep 06 10:49:12.189380 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:12.189341 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk"] Sep 06 10:49:12.192658 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:12.192638 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aec3c2bf-7f7947f7c7-thkmk"] Sep 06 10:49:12.926496 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:12.926464 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed489e30-b06a-4123-8c7a-6eda7e1f23eb" path="/var/lib/kubelet/pods/ed489e30-b06a-4123-8c7a-6eda7e1f23eb/volumes" Sep 06 10:49:12.926824 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:12.926811 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa5d66a5-9401-4634-ab85-dee8ed854447" path="/var/lib/kubelet/pods/fa5d66a5-9401-4634-ab85-dee8ed854447/volumes" Sep 06 10:49:14.449142 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:14.449091 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:49:19.434880 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:19.434840 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz"] Sep 06 10:49:19.529463 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:19.529429 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:49:34.455082 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.454972 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" podUID="498da61b-8c07-44ae-9847-df1791622a74" containerName="main" containerID="cri-o://292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b" gracePeriod=45 Sep 06 10:49:34.707900 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.707841 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:49:34.786712 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.786679 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-home\") pod \"498da61b-8c07-44ae-9847-df1791622a74\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " Sep 06 10:49:34.786890 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.786729 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-dshm\") pod \"498da61b-8c07-44ae-9847-df1791622a74\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " Sep 06 10:49:34.786890 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.786830 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-model-cache\") pod \"498da61b-8c07-44ae-9847-df1791622a74\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " Sep 06 10:49:34.786890 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.786850 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqw76\" (UniqueName: \"kubernetes.io/projected/498da61b-8c07-44ae-9847-df1791622a74-kube-api-access-zqw76\") pod \"498da61b-8c07-44ae-9847-df1791622a74\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " Sep 06 10:49:34.786890 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.786877 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-tmp-dir\") pod \"498da61b-8c07-44ae-9847-df1791622a74\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " Sep 06 10:49:34.787133 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.786907 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/498da61b-8c07-44ae-9847-df1791622a74-tls-certs\") pod \"498da61b-8c07-44ae-9847-df1791622a74\" (UID: \"498da61b-8c07-44ae-9847-df1791622a74\") " Sep 06 10:49:34.787133 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.787098 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-home" (OuterVolumeSpecName: "home") pod "498da61b-8c07-44ae-9847-df1791622a74" (UID: "498da61b-8c07-44ae-9847-df1791622a74"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:49:34.787257 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.787109 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-model-cache" (OuterVolumeSpecName: "model-cache") pod "498da61b-8c07-44ae-9847-df1791622a74" (UID: "498da61b-8c07-44ae-9847-df1791622a74"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:49:34.787257 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.787214 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "498da61b-8c07-44ae-9847-df1791622a74" (UID: "498da61b-8c07-44ae-9847-df1791622a74"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:49:34.787373 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.787294 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-home\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:49:34.787373 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.787313 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-model-cache\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:49:34.789002 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.788965 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-dshm" (OuterVolumeSpecName: "dshm") pod "498da61b-8c07-44ae-9847-df1791622a74" (UID: "498da61b-8c07-44ae-9847-df1791622a74"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:49:34.789214 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.789097 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/498da61b-8c07-44ae-9847-df1791622a74-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "498da61b-8c07-44ae-9847-df1791622a74" (UID: "498da61b-8c07-44ae-9847-df1791622a74"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 10:49:34.789346 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.789324 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/498da61b-8c07-44ae-9847-df1791622a74-kube-api-access-zqw76" (OuterVolumeSpecName: "kube-api-access-zqw76") pod "498da61b-8c07-44ae-9847-df1791622a74" (UID: "498da61b-8c07-44ae-9847-df1791622a74"). InnerVolumeSpecName "kube-api-access-zqw76". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:49:34.888723 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.888675 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-dshm\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:49:34.888723 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.888718 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqw76\" (UniqueName: \"kubernetes.io/projected/498da61b-8c07-44ae-9847-df1791622a74-kube-api-access-zqw76\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:49:34.888723 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.888732 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/498da61b-8c07-44ae-9847-df1791622a74-tmp-dir\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:49:34.888955 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:34.888745 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/498da61b-8c07-44ae-9847-df1791622a74-tls-certs\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:49:35.306472 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:35.306430 2574 generic.go:356] "Generic (PLEG): container finished" podID="498da61b-8c07-44ae-9847-df1791622a74" containerID="292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b" exitCode=0 Sep 06 10:49:35.306650 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:35.306476 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" event={"ID":"498da61b-8c07-44ae-9847-df1791622a74","Type":"ContainerDied","Data":"292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b"} Sep 06 10:49:35.306650 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:35.306490 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" Sep 06 10:49:35.306650 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:35.306515 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz" event={"ID":"498da61b-8c07-44ae-9847-df1791622a74","Type":"ContainerDied","Data":"545ab347c34ca265907ac7cbe1b3c77c3670b96fbe70f6f94d5cce9edcd7fb74"} Sep 06 10:49:35.306650 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:35.306539 2574 scope.go:117] "RemoveContainer" containerID="292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b" Sep 06 10:49:35.315525 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:35.315506 2574 scope.go:117] "RemoveContainer" containerID="292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b" Sep 06 10:49:35.315778 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:49:35.315755 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b\": container with ID starting with 292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b not found: ID does not exist" containerID="292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b" Sep 06 10:49:35.315845 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:35.315788 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b"} err="failed to get container status \"292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b\": rpc error: code = NotFound desc = could not find container \"292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b\": container with ID starting with 292257b90d95b02291592727c3dc09de21246362af1da130309f42fd905ce71b not found: ID does not exist" Sep 06 10:49:35.322335 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:35.322312 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz"] Sep 06 10:49:35.324716 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:35.324695 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-aec3c2bf/e2e-lifecycle-model-aec3c2bf-kserve-5ffb586b7-vm2qz"] Sep 06 10:49:36.926948 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:36.926915 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="498da61b-8c07-44ae-9847-df1791622a74" path="/var/lib/kubelet/pods/498da61b-8c07-44ae-9847-df1791622a74/volumes" Sep 06 10:49:55.569359 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:55.569323 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:49:56.293965 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:49:56.293932 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:02.000849 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:02.000817 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:02.100419 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:02.100373 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:02.259546 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:02.259463 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:03.462680 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:03.462639 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:03.562732 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:03.562695 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:03.663052 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:03.663023 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:04.237951 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:04.237921 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:04.336056 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:04.336019 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:04.437185 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:04.437152 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:05.670389 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:05.670342 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:11.359747 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:11.359714 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:12.694184 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:12.694144 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:17.821200 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:17.821147 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:17.881257 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:17.881213 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:17.952300 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:17.952270 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:18.051998 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:18.051963 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:25.367067 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:25.367025 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:26.251881 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:26.251845 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:28.485239 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:28.485210 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:28.705071 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:28.705029 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:39.241548 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:39.241500 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:49.644684 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:49.644643 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:55.340529 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:55.340483 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:55.401883 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:55.401849 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:55.696796 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:55.696761 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:55.992085 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:55.992011 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:56.199098 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:56.199061 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:58.934349 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:58.934316 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:50:59.035136 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:50:59.035101 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:00.089885 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.089851 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:00.147681 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.147650 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:00.457887 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.457859 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:00.571305 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.571272 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:00.677027 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.676995 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:00.717170 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.717071 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j"] Sep 06 10:51:00.717524 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.717510 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed489e30-b06a-4123-8c7a-6eda7e1f23eb" containerName="revoke-keys" Sep 06 10:51:00.717571 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.717527 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed489e30-b06a-4123-8c7a-6eda7e1f23eb" containerName="revoke-keys" Sep 06 10:51:00.717571 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.717540 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="498da61b-8c07-44ae-9847-df1791622a74" containerName="main" Sep 06 10:51:00.717571 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.717546 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="498da61b-8c07-44ae-9847-df1791622a74" containerName="main" Sep 06 10:51:00.717571 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.717564 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa5d66a5-9401-4634-ab85-dee8ed854447" containerName="payload-processing" Sep 06 10:51:00.717571 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.717570 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa5d66a5-9401-4634-ab85-dee8ed854447" containerName="payload-processing" Sep 06 10:51:00.717743 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.717643 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa5d66a5-9401-4634-ab85-dee8ed854447" containerName="payload-processing" Sep 06 10:51:00.717743 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.717653 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed489e30-b06a-4123-8c7a-6eda7e1f23eb" containerName="revoke-keys" Sep 06 10:51:00.717743 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.717659 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="498da61b-8c07-44ae-9847-df1791622a74" containerName="main" Sep 06 10:51:00.720846 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.720830 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" Sep 06 10:51:00.722876 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.722855 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-866a8b\"" Sep 06 10:51:00.722979 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.722950 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-866a8b-dockercfg-x9rqp\"" Sep 06 10:51:00.729796 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.729775 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j"] Sep 06 10:51:00.779732 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.779699 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e47ca4d-b7bd-4154-be28-c77f3db5320c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j\" (UID: \"0e47ca4d-b7bd-4154-be28-c77f3db5320c\") " pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" Sep 06 10:51:00.780058 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.780007 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwp8l\" (UniqueName: \"kubernetes.io/projected/0e47ca4d-b7bd-4154-be28-c77f3db5320c-kube-api-access-jwp8l\") pod \"payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j\" (UID: \"0e47ca4d-b7bd-4154-be28-c77f3db5320c\") " pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" Sep 06 10:51:00.880580 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.880541 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwp8l\" (UniqueName: \"kubernetes.io/projected/0e47ca4d-b7bd-4154-be28-c77f3db5320c-kube-api-access-jwp8l\") pod \"payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j\" (UID: \"0e47ca4d-b7bd-4154-be28-c77f3db5320c\") " pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" Sep 06 10:51:00.880855 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.880670 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e47ca4d-b7bd-4154-be28-c77f3db5320c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j\" (UID: \"0e47ca4d-b7bd-4154-be28-c77f3db5320c\") " pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" Sep 06 10:51:00.881379 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.881360 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e47ca4d-b7bd-4154-be28-c77f3db5320c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j\" (UID: \"0e47ca4d-b7bd-4154-be28-c77f3db5320c\") " pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" Sep 06 10:51:00.887808 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:00.887778 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwp8l\" (UniqueName: \"kubernetes.io/projected/0e47ca4d-b7bd-4154-be28-c77f3db5320c-kube-api-access-jwp8l\") pod \"payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j\" (UID: \"0e47ca4d-b7bd-4154-be28-c77f3db5320c\") " pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" Sep 06 10:51:01.032279 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:01.032184 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" Sep 06 10:51:01.302923 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:01.302893 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j"] Sep 06 10:51:01.304540 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:51:01.304516 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e47ca4d_b7bd_4154_be28_c77f3db5320c.slice/crio-ed30201af47d38177a18e0c885f406e5b4f22f8440c8d9d02f3b75fe08fcfce3 WatchSource:0}: Error finding container ed30201af47d38177a18e0c885f406e5b4f22f8440c8d9d02f3b75fe08fcfce3: Status 404 returned error can't find the container with id ed30201af47d38177a18e0c885f406e5b4f22f8440c8d9d02f3b75fe08fcfce3 Sep 06 10:51:01.817723 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:01.817691 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" event={"ID":"0e47ca4d-b7bd-4154-be28-c77f3db5320c","Type":"ContainerStarted","Data":"4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8"} Sep 06 10:51:01.817923 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:01.817740 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" event={"ID":"0e47ca4d-b7bd-4154-be28-c77f3db5320c","Type":"ContainerStarted","Data":"ed30201af47d38177a18e0c885f406e5b4f22f8440c8d9d02f3b75fe08fcfce3"} Sep 06 10:51:01.817923 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:01.817829 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" Sep 06 10:51:01.832407 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:01.832360 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" podStartSLOduration=1.832345324 podStartE2EDuration="1.832345324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:51:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:51:01.831673943 +0000 UTC m=+1255.414163575" watchObservedRunningTime="2026-09-06 10:51:01.832345324 +0000 UTC m=+1255.414834954" Sep 06 10:51:12.826156 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:12.826104 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" Sep 06 10:51:16.432747 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:16.432708 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:16.530675 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:16.530622 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:21.049853 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:21.049819 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:23.930606 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:23.930564 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:24.409950 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:24.409858 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:24.717410 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:24.717380 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:25.653895 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:25.653853 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:28.143046 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:28.143007 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:28.243283 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:28.243246 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:29.279991 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:29.279952 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:29.351991 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:29.351945 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:29.784315 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:29.784281 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:29.870600 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:29.870562 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg"] Sep 06 10:51:29.871532 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:29.871507 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:29.874347 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:29.874330 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" Sep 06 10:51:29.876762 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:29.876743 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-b83cfe\"" Sep 06 10:51:29.889911 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:29.889883 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg"] Sep 06 10:51:29.982874 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:29.982845 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:30.055876 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:30.055789 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx7w8\" (UniqueName: \"kubernetes.io/projected/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc-kube-api-access-fx7w8\") pod \"payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg\" (UID: \"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" Sep 06 10:51:30.056055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:30.055928 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg\" (UID: \"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" Sep 06 10:51:30.157158 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:30.157090 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx7w8\" (UniqueName: \"kubernetes.io/projected/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc-kube-api-access-fx7w8\") pod \"payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg\" (UID: \"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" Sep 06 10:51:30.157344 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:30.157242 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg\" (UID: \"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" Sep 06 10:51:30.157989 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:30.157970 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg\" (UID: \"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" Sep 06 10:51:30.164420 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:30.164389 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx7w8\" (UniqueName: \"kubernetes.io/projected/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc-kube-api-access-fx7w8\") pod \"payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg\" (UID: \"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" Sep 06 10:51:30.183786 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:30.183760 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" Sep 06 10:51:30.456927 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:30.456896 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg"] Sep 06 10:51:30.458535 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:51:30.458507 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f1b092a_20e3_426c_a02a_8e9ffeafcdcc.slice/crio-aec8f49360e1424b5cc868458b0450bcdb44d74db03cd600ebc885a3afbd6895 WatchSource:0}: Error finding container aec8f49360e1424b5cc868458b0450bcdb44d74db03cd600ebc885a3afbd6895: Status 404 returned error can't find the container with id aec8f49360e1424b5cc868458b0450bcdb44d74db03cd600ebc885a3afbd6895 Sep 06 10:51:30.992219 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:30.992175 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" event={"ID":"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc","Type":"ContainerStarted","Data":"adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5"} Sep 06 10:51:30.992429 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:30.992239 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" event={"ID":"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc","Type":"ContainerStarted","Data":"aec8f49360e1424b5cc868458b0450bcdb44d74db03cd600ebc885a3afbd6895"} Sep 06 10:51:30.992429 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:30.992264 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" Sep 06 10:51:31.008132 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:31.008071 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" podStartSLOduration=2.008057484 podStartE2EDuration="2.008057484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:51:31.006182523 +0000 UTC m=+1284.588672157" watchObservedRunningTime="2026-09-06 10:51:31.008057484 +0000 UTC m=+1284.590547115" Sep 06 10:51:42.000580 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:42.000545 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" Sep 06 10:51:44.045580 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:44.045542 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:48.239100 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:48.239062 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:50.227672 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:50.227634 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:53.056008 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:53.055975 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:55.142651 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:55.142616 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:55.895170 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:55.895133 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp"] Sep 06 10:51:55.898896 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:55.898879 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp" Sep 06 10:51:55.901268 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:55.901241 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 10:51:55.901995 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:55.901967 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\"" Sep 06 10:51:55.904992 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:55.904968 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp"] Sep 06 10:51:55.995075 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:55.995033 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0b8c67-30a2-48cb-b550-c71aecabeb8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp\" (UID: \"1a0b8c67-30a2-48cb-b550-c71aecabeb8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp" Sep 06 10:51:56.015112 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:56.015078 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:51:56.096133 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:56.096082 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0b8c67-30a2-48cb-b550-c71aecabeb8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp\" (UID: \"1a0b8c67-30a2-48cb-b550-c71aecabeb8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp" Sep 06 10:51:56.096952 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:56.096929 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0b8c67-30a2-48cb-b550-c71aecabeb8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp\" (UID: \"1a0b8c67-30a2-48cb-b550-c71aecabeb8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp" Sep 06 10:51:56.212407 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:56.212372 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp" Sep 06 10:51:56.483524 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:56.483491 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp"] Sep 06 10:51:56.485084 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:51:56.485055 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0b8c67_30a2_48cb_b550_c71aecabeb8a.slice/crio-d43b9a0821f2b0656de16532f7e7b4bf0bcbb906df751395a113c7ff4e613dd9 WatchSource:0}: Error finding container d43b9a0821f2b0656de16532f7e7b4bf0bcbb906df751395a113c7ff4e613dd9: Status 404 returned error can't find the container with id d43b9a0821f2b0656de16532f7e7b4bf0bcbb906df751395a113c7ff4e613dd9 Sep 06 10:51:57.152909 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:57.152878 2574 generic.go:356] "Generic (PLEG): container finished" podID="1a0b8c67-30a2-48cb-b550-c71aecabeb8a" containerID="87cdb85a611419149bc3550ed96e57348bdba0b5b3b6a6e87db2bede8fc8e5ac" exitCode=0 Sep 06 10:51:57.153055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:57.152920 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp" event={"ID":"1a0b8c67-30a2-48cb-b550-c71aecabeb8a","Type":"ContainerDied","Data":"87cdb85a611419149bc3550ed96e57348bdba0b5b3b6a6e87db2bede8fc8e5ac"} Sep 06 10:51:57.153055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:57.152958 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp" event={"ID":"1a0b8c67-30a2-48cb-b550-c71aecabeb8a","Type":"ContainerStarted","Data":"d43b9a0821f2b0656de16532f7e7b4bf0bcbb906df751395a113c7ff4e613dd9"} Sep 06 10:51:58.301062 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:58.301039 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp" Sep 06 10:51:58.419114 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:58.419088 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0b8c67-30a2-48cb-b550-c71aecabeb8a-maas-api-service-ca\") pod \"1a0b8c67-30a2-48cb-b550-c71aecabeb8a\" (UID: \"1a0b8c67-30a2-48cb-b550-c71aecabeb8a\") " Sep 06 10:51:58.419423 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:58.419399 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a0b8c67-30a2-48cb-b550-c71aecabeb8a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1a0b8c67-30a2-48cb-b550-c71aecabeb8a" (UID: "1a0b8c67-30a2-48cb-b550-c71aecabeb8a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:51:58.520167 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:58.520104 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0b8c67-30a2-48cb-b550-c71aecabeb8a-maas-api-service-ca\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:51:59.166798 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:59.166761 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp" Sep 06 10:51:59.166953 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:59.166756 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp" event={"ID":"1a0b8c67-30a2-48cb-b550-c71aecabeb8a","Type":"ContainerDied","Data":"d43b9a0821f2b0656de16532f7e7b4bf0bcbb906df751395a113c7ff4e613dd9"} Sep 06 10:51:59.166953 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:51:59.166887 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d43b9a0821f2b0656de16532f7e7b4bf0bcbb906df751395a113c7ff4e613dd9" Sep 06 10:52:07.108133 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:07.108101 2574 scope.go:117] "RemoveContainer" containerID="9c89bd483e739ce6596f1891b1970b9d31e823b01859db4c31b41e03765f97f4" Sep 06 10:52:08.639541 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:08.639506 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:09.529647 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:09.529619 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:11.357270 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.357217 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:11.384709 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.384676 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j"] Sep 06 10:52:11.384959 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.384919 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" podUID="0e47ca4d-b7bd-4154-be28-c77f3db5320c" containerName="payload-processing" containerID="cri-o://4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8" gracePeriod=30 Sep 06 10:52:11.457893 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.457861 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:11.536075 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.536039 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp"] Sep 06 10:52:11.537442 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.537405 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:11.538826 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.538804 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-866a8b-ckmrp"] Sep 06 10:52:11.603791 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.603759 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:11.638992 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.638968 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" Sep 06 10:52:11.742583 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.742547 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwp8l\" (UniqueName: \"kubernetes.io/projected/0e47ca4d-b7bd-4154-be28-c77f3db5320c-kube-api-access-jwp8l\") pod \"0e47ca4d-b7bd-4154-be28-c77f3db5320c\" (UID: \"0e47ca4d-b7bd-4154-be28-c77f3db5320c\") " Sep 06 10:52:11.742763 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.742617 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e47ca4d-b7bd-4154-be28-c77f3db5320c-plugins-config-volume\") pod \"0e47ca4d-b7bd-4154-be28-c77f3db5320c\" (UID: \"0e47ca4d-b7bd-4154-be28-c77f3db5320c\") " Sep 06 10:52:11.742964 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.742943 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e47ca4d-b7bd-4154-be28-c77f3db5320c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0e47ca4d-b7bd-4154-be28-c77f3db5320c" (UID: "0e47ca4d-b7bd-4154-be28-c77f3db5320c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:52:11.744642 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.744620 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e47ca4d-b7bd-4154-be28-c77f3db5320c-kube-api-access-jwp8l" (OuterVolumeSpecName: "kube-api-access-jwp8l") pod "0e47ca4d-b7bd-4154-be28-c77f3db5320c" (UID: "0e47ca4d-b7bd-4154-be28-c77f3db5320c"). InnerVolumeSpecName "kube-api-access-jwp8l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:52:11.844253 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.844224 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwp8l\" (UniqueName: \"kubernetes.io/projected/0e47ca4d-b7bd-4154-be28-c77f3db5320c-kube-api-access-jwp8l\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:52:11.844253 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.844250 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e47ca4d-b7bd-4154-be28-c77f3db5320c-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:52:11.885285 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:11.885183 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:12.239642 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:12.239608 2574 generic.go:356] "Generic (PLEG): container finished" podID="0e47ca4d-b7bd-4154-be28-c77f3db5320c" containerID="4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8" exitCode=0 Sep 06 10:52:12.239846 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:12.239649 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" Sep 06 10:52:12.239846 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:12.239678 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" event={"ID":"0e47ca4d-b7bd-4154-be28-c77f3db5320c","Type":"ContainerDied","Data":"4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8"} Sep 06 10:52:12.239846 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:12.239728 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j" event={"ID":"0e47ca4d-b7bd-4154-be28-c77f3db5320c","Type":"ContainerDied","Data":"ed30201af47d38177a18e0c885f406e5b4f22f8440c8d9d02f3b75fe08fcfce3"} Sep 06 10:52:12.239846 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:12.239756 2574 scope.go:117] "RemoveContainer" containerID="4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8" Sep 06 10:52:12.249568 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:12.249548 2574 scope.go:117] "RemoveContainer" containerID="4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8" Sep 06 10:52:12.249822 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:52:12.249801 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8\": container with ID starting with 4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8 not found: ID does not exist" containerID="4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8" Sep 06 10:52:12.249882 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:12.249830 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8"} err="failed to get container status \"4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8\": rpc error: code = NotFound desc = could not find container \"4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8\": container with ID starting with 4024df608f9a5459debb79c876dda13b4a00ba8d8a6ef5728f2cc49f3bd6eae8 not found: ID does not exist" Sep 06 10:52:12.261070 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:12.261024 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j"] Sep 06 10:52:12.264060 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:12.264036 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-866a8b-664d66cbc7-nsc4j"] Sep 06 10:52:12.927140 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:12.927103 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e47ca4d-b7bd-4154-be28-c77f3db5320c" path="/var/lib/kubelet/pods/0e47ca4d-b7bd-4154-be28-c77f3db5320c/volumes" Sep 06 10:52:12.927520 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:12.927506 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a0b8c67-30a2-48cb-b550-c71aecabeb8a" path="/var/lib/kubelet/pods/1a0b8c67-30a2-48cb-b550-c71aecabeb8a/volumes" Sep 06 10:52:16.958706 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:16.958669 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:17.778418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:17.778382 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:17.837132 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:17.837100 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:18.031502 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.031410 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:18.372673 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.372582 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9"] Sep 06 10:52:18.373041 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.373026 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a0b8c67-30a2-48cb-b550-c71aecabeb8a" containerName="revoke-keys" Sep 06 10:52:18.373097 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.373042 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a0b8c67-30a2-48cb-b550-c71aecabeb8a" containerName="revoke-keys" Sep 06 10:52:18.373097 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.373059 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e47ca4d-b7bd-4154-be28-c77f3db5320c" containerName="payload-processing" Sep 06 10:52:18.373097 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.373065 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e47ca4d-b7bd-4154-be28-c77f3db5320c" containerName="payload-processing" Sep 06 10:52:18.373253 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.373153 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a0b8c67-30a2-48cb-b550-c71aecabeb8a" containerName="revoke-keys" Sep 06 10:52:18.373253 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.373168 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e47ca4d-b7bd-4154-be28-c77f3db5320c" containerName="payload-processing" Sep 06 10:52:18.377757 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.377737 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9" Sep 06 10:52:18.380119 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.380087 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\"" Sep 06 10:52:18.380119 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.380087 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 10:52:18.382409 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.382379 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9"] Sep 06 10:52:18.404491 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.404466 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6d3da3d-b796-4fd3-bf7d-677ebb6963f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9\" (UID: \"f6d3da3d-b796-4fd3-bf7d-677ebb6963f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9" Sep 06 10:52:18.467448 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.467410 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:18.505724 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.505691 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6d3da3d-b796-4fd3-bf7d-677ebb6963f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9\" (UID: \"f6d3da3d-b796-4fd3-bf7d-677ebb6963f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9" Sep 06 10:52:18.506339 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.506319 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6d3da3d-b796-4fd3-bf7d-677ebb6963f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9\" (UID: \"f6d3da3d-b796-4fd3-bf7d-677ebb6963f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9" Sep 06 10:52:18.689902 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.689857 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9" Sep 06 10:52:18.959933 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:18.959907 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9"] Sep 06 10:52:18.962152 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:52:18.962097 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d3da3d_b796_4fd3_bf7d_677ebb6963f6.slice/crio-05eba0730688e515013271da4b3406f9234a490449d3f8709b0c3e6a72a11213 WatchSource:0}: Error finding container 05eba0730688e515013271da4b3406f9234a490449d3f8709b0c3e6a72a11213: Status 404 returned error can't find the container with id 05eba0730688e515013271da4b3406f9234a490449d3f8709b0c3e6a72a11213 Sep 06 10:52:19.285758 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:19.285726 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9" event={"ID":"f6d3da3d-b796-4fd3-bf7d-677ebb6963f6","Type":"ContainerStarted","Data":"c350935930ac11ca1d8a96bfbd424c51ce46412be1f109e251541d3cecc6ea7e"} Sep 06 10:52:19.286086 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:19.285778 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9" event={"ID":"f6d3da3d-b796-4fd3-bf7d-677ebb6963f6","Type":"ContainerStarted","Data":"05eba0730688e515013271da4b3406f9234a490449d3f8709b0c3e6a72a11213"} Sep 06 10:52:19.300008 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:19.299965 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9" podStartSLOduration=1.299950429 podStartE2EDuration="1.299950429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:52:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:52:19.298474373 +0000 UTC m=+1332.880964004" watchObservedRunningTime="2026-09-06 10:52:19.299950429 +0000 UTC m=+1332.882440060" Sep 06 10:52:20.292797 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:20.292762 2574 generic.go:356] "Generic (PLEG): container finished" podID="f6d3da3d-b796-4fd3-bf7d-677ebb6963f6" containerID="c350935930ac11ca1d8a96bfbd424c51ce46412be1f109e251541d3cecc6ea7e" exitCode=0 Sep 06 10:52:20.293182 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:20.292839 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9" event={"ID":"f6d3da3d-b796-4fd3-bf7d-677ebb6963f6","Type":"ContainerDied","Data":"c350935930ac11ca1d8a96bfbd424c51ce46412be1f109e251541d3cecc6ea7e"} Sep 06 10:52:21.431566 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:21.431537 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9" Sep 06 10:52:21.533210 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:21.533178 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6d3da3d-b796-4fd3-bf7d-677ebb6963f6-maas-api-service-ca\") pod \"f6d3da3d-b796-4fd3-bf7d-677ebb6963f6\" (UID: \"f6d3da3d-b796-4fd3-bf7d-677ebb6963f6\") " Sep 06 10:52:21.533571 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:21.533544 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6d3da3d-b796-4fd3-bf7d-677ebb6963f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f6d3da3d-b796-4fd3-bf7d-677ebb6963f6" (UID: "f6d3da3d-b796-4fd3-bf7d-677ebb6963f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:52:21.634433 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:21.634351 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6d3da3d-b796-4fd3-bf7d-677ebb6963f6-maas-api-service-ca\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:52:22.306094 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:22.306065 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9" Sep 06 10:52:22.306288 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:22.306111 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9" event={"ID":"f6d3da3d-b796-4fd3-bf7d-677ebb6963f6","Type":"ContainerDied","Data":"05eba0730688e515013271da4b3406f9234a490449d3f8709b0c3e6a72a11213"} Sep 06 10:52:22.306288 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:22.306180 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05eba0730688e515013271da4b3406f9234a490449d3f8709b0c3e6a72a11213" Sep 06 10:52:27.553661 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:27.553631 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:33.962073 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:33.961984 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:33.994633 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:33.994597 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg"] Sep 06 10:52:33.994871 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:33.994834 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" podUID="3f1b092a-20e3-426c-a02a-8e9ffeafcdcc" containerName="payload-pre-processing" containerID="cri-o://adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5" gracePeriod=30 Sep 06 10:52:34.062486 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.062456 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:34.150520 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.150227 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9"] Sep 06 10:52:34.152056 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.152028 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:34.152983 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.152961 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b83cfe-k7dz9"] Sep 06 10:52:34.209037 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.209006 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:34.249548 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.249501 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" Sep 06 10:52:34.350309 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.350277 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fx7w8\" (UniqueName: \"kubernetes.io/projected/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc-kube-api-access-fx7w8\") pod \"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc\" (UID: \"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc\") " Sep 06 10:52:34.350505 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.350362 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc-plugins-config-volume\") pod \"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc\" (UID: \"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc\") " Sep 06 10:52:34.350725 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.350695 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f1b092a-20e3-426c-a02a-8e9ffeafcdcc" (UID: "3f1b092a-20e3-426c-a02a-8e9ffeafcdcc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:52:34.352412 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.352388 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc-kube-api-access-fx7w8" (OuterVolumeSpecName: "kube-api-access-fx7w8") pod "3f1b092a-20e3-426c-a02a-8e9ffeafcdcc" (UID: "3f1b092a-20e3-426c-a02a-8e9ffeafcdcc"). InnerVolumeSpecName "kube-api-access-fx7w8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:52:34.377858 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.377826 2574 generic.go:356] "Generic (PLEG): container finished" podID="3f1b092a-20e3-426c-a02a-8e9ffeafcdcc" containerID="adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5" exitCode=0 Sep 06 10:52:34.377972 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.377877 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" Sep 06 10:52:34.377972 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.377912 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" event={"ID":"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc","Type":"ContainerDied","Data":"adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5"} Sep 06 10:52:34.378050 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.377989 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg" event={"ID":"3f1b092a-20e3-426c-a02a-8e9ffeafcdcc","Type":"ContainerDied","Data":"aec8f49360e1424b5cc868458b0450bcdb44d74db03cd600ebc885a3afbd6895"} Sep 06 10:52:34.378050 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.378013 2574 scope.go:117] "RemoveContainer" containerID="adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5" Sep 06 10:52:34.389503 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.389485 2574 scope.go:117] "RemoveContainer" containerID="adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5" Sep 06 10:52:34.389777 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:52:34.389757 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5\": container with ID starting with adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5 not found: ID does not exist" containerID="adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5" Sep 06 10:52:34.389856 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.389785 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5"} err="failed to get container status \"adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5\": rpc error: code = NotFound desc = could not find container \"adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5\": container with ID starting with adad5ba58262c36549689e07afe7f1f6510a275381d4752b5436b85245451ac5 not found: ID does not exist" Sep 06 10:52:34.402295 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.402182 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg"] Sep 06 10:52:34.404934 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.404908 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b83cfe-85bd647b7d-qtxkg"] Sep 06 10:52:34.451785 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.451758 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fx7w8\" (UniqueName: \"kubernetes.io/projected/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc-kube-api-access-fx7w8\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:52:34.451785 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.451782 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:52:34.518489 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.518407 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:34.926822 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.926789 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f1b092a-20e3-426c-a02a-8e9ffeafcdcc" path="/var/lib/kubelet/pods/3f1b092a-20e3-426c-a02a-8e9ffeafcdcc/volumes" Sep 06 10:52:34.927211 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:34.927196 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6d3da3d-b796-4fd3-bf7d-677ebb6963f6" path="/var/lib/kubelet/pods/f6d3da3d-b796-4fd3-bf7d-677ebb6963f6/volumes" Sep 06 10:52:40.183835 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.183798 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:40.283373 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.283336 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:40.439315 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.439226 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:40.767908 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.767813 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc"] Sep 06 10:52:40.768417 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.768398 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f1b092a-20e3-426c-a02a-8e9ffeafcdcc" containerName="payload-pre-processing" Sep 06 10:52:40.768527 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.768419 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f1b092a-20e3-426c-a02a-8e9ffeafcdcc" containerName="payload-pre-processing" Sep 06 10:52:40.768527 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.768433 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6d3da3d-b796-4fd3-bf7d-677ebb6963f6" containerName="revoke-keys" Sep 06 10:52:40.768527 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.768442 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6d3da3d-b796-4fd3-bf7d-677ebb6963f6" containerName="revoke-keys" Sep 06 10:52:40.768683 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.768527 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f1b092a-20e3-426c-a02a-8e9ffeafcdcc" containerName="payload-pre-processing" Sep 06 10:52:40.768683 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.768540 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6d3da3d-b796-4fd3-bf7d-677ebb6963f6" containerName="revoke-keys" Sep 06 10:52:40.771792 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.771772 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc" Sep 06 10:52:40.774837 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.774811 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\"" Sep 06 10:52:40.775050 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.774853 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 10:52:40.778788 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.778686 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc"] Sep 06 10:52:40.917149 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.917089 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e528eef8-5955-418b-9967-f30083dc5d40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc\" (UID: \"e528eef8-5955-418b-9967-f30083dc5d40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc" Sep 06 10:52:40.943619 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:40.943590 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:41.018101 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:41.018019 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e528eef8-5955-418b-9967-f30083dc5d40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc\" (UID: \"e528eef8-5955-418b-9967-f30083dc5d40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc" Sep 06 10:52:41.018712 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:41.018692 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e528eef8-5955-418b-9967-f30083dc5d40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc\" (UID: \"e528eef8-5955-418b-9967-f30083dc5d40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc" Sep 06 10:52:41.083996 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:41.083967 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc" Sep 06 10:52:41.354483 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:41.354453 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc"] Sep 06 10:52:41.355709 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:52:41.355683 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode528eef8_5955_418b_9967_f30083dc5d40.slice/crio-1b8af4daf5a5e9bce7664ec5bf6e6fd4da206f9e026f6e989d9b64702f6094ac WatchSource:0}: Error finding container 1b8af4daf5a5e9bce7664ec5bf6e6fd4da206f9e026f6e989d9b64702f6094ac: Status 404 returned error can't find the container with id 1b8af4daf5a5e9bce7664ec5bf6e6fd4da206f9e026f6e989d9b64702f6094ac Sep 06 10:52:41.423383 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:41.423354 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc" event={"ID":"e528eef8-5955-418b-9967-f30083dc5d40","Type":"ContainerStarted","Data":"1b8af4daf5a5e9bce7664ec5bf6e6fd4da206f9e026f6e989d9b64702f6094ac"} Sep 06 10:52:42.430966 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:42.430934 2574 generic.go:356] "Generic (PLEG): container finished" podID="e528eef8-5955-418b-9967-f30083dc5d40" containerID="0fde5f9d646c1ae648caa0bf1ff418ae044be0e7d282c4a22de93f410f8b7b2a" exitCode=0 Sep 06 10:52:42.431400 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:42.431030 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc" event={"ID":"e528eef8-5955-418b-9967-f30083dc5d40","Type":"ContainerDied","Data":"0fde5f9d646c1ae648caa0bf1ff418ae044be0e7d282c4a22de93f410f8b7b2a"} Sep 06 10:52:43.587317 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:43.587293 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc" Sep 06 10:52:43.745273 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:43.745171 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e528eef8-5955-418b-9967-f30083dc5d40-maas-api-service-ca\") pod \"e528eef8-5955-418b-9967-f30083dc5d40\" (UID: \"e528eef8-5955-418b-9967-f30083dc5d40\") " Sep 06 10:52:43.745540 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:43.745516 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e528eef8-5955-418b-9967-f30083dc5d40-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e528eef8-5955-418b-9967-f30083dc5d40" (UID: "e528eef8-5955-418b-9967-f30083dc5d40"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:52:43.846452 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:43.846421 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e528eef8-5955-418b-9967-f30083dc5d40-maas-api-service-ca\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:52:44.444565 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:44.444537 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc" Sep 06 10:52:44.444565 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:44.444551 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc" event={"ID":"e528eef8-5955-418b-9967-f30083dc5d40","Type":"ContainerDied","Data":"1b8af4daf5a5e9bce7664ec5bf6e6fd4da206f9e026f6e989d9b64702f6094ac"} Sep 06 10:52:44.444779 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:44.444604 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b8af4daf5a5e9bce7664ec5bf6e6fd4da206f9e026f6e989d9b64702f6094ac" Sep 06 10:52:44.852374 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:44.852298 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:45.884219 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:45.884187 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:45.937065 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:45.937032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:51.183567 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.183532 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:51.211771 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.211740 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725"] Sep 06 10:52:51.211965 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.211940 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" podUID="2fcd3188-40ca-4863-b0cb-12780020c1f4" containerName="payload-processing" containerID="cri-o://7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3" gracePeriod=30 Sep 06 10:52:51.286798 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.286765 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:51.370621 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.370592 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc"] Sep 06 10:52:51.371283 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.371257 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:51.374170 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.374147 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-72cb16-sc6fc"] Sep 06 10:52:51.425284 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.425256 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:51.475506 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.475473 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" Sep 06 10:52:51.485480 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.485452 2574 generic.go:356] "Generic (PLEG): container finished" podID="2fcd3188-40ca-4863-b0cb-12780020c1f4" containerID="7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3" exitCode=0 Sep 06 10:52:51.485610 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.485488 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" Sep 06 10:52:51.485610 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.485506 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" event={"ID":"2fcd3188-40ca-4863-b0cb-12780020c1f4","Type":"ContainerDied","Data":"7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3"} Sep 06 10:52:51.485610 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.485545 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725" event={"ID":"2fcd3188-40ca-4863-b0cb-12780020c1f4","Type":"ContainerDied","Data":"43788f4816b1b7b426d3efba57d2ee38a60ce81b92a03acebf1db154c5a44851"} Sep 06 10:52:51.485610 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.485567 2574 scope.go:117] "RemoveContainer" containerID="7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3" Sep 06 10:52:51.496753 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.496730 2574 scope.go:117] "RemoveContainer" containerID="7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3" Sep 06 10:52:51.497441 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:52:51.497118 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3\": container with ID starting with 7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3 not found: ID does not exist" containerID="7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3" Sep 06 10:52:51.497441 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.497351 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3"} err="failed to get container status \"7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3\": rpc error: code = NotFound desc = could not find container \"7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3\": container with ID starting with 7c9b380002708827eb8594fc075a61dd997d25600b44c4728aeb48b5ad2230a3 not found: ID does not exist" Sep 06 10:52:51.617344 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.617314 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fcd3188-40ca-4863-b0cb-12780020c1f4-plugins-config-volume\") pod \"2fcd3188-40ca-4863-b0cb-12780020c1f4\" (UID: \"2fcd3188-40ca-4863-b0cb-12780020c1f4\") " Sep 06 10:52:51.617344 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.617351 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vbsz\" (UniqueName: \"kubernetes.io/projected/2fcd3188-40ca-4863-b0cb-12780020c1f4-kube-api-access-2vbsz\") pod \"2fcd3188-40ca-4863-b0cb-12780020c1f4\" (UID: \"2fcd3188-40ca-4863-b0cb-12780020c1f4\") " Sep 06 10:52:51.617699 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.617671 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fcd3188-40ca-4863-b0cb-12780020c1f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2fcd3188-40ca-4863-b0cb-12780020c1f4" (UID: "2fcd3188-40ca-4863-b0cb-12780020c1f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:52:51.619520 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.619497 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fcd3188-40ca-4863-b0cb-12780020c1f4-kube-api-access-2vbsz" (OuterVolumeSpecName: "kube-api-access-2vbsz") pod "2fcd3188-40ca-4863-b0cb-12780020c1f4" (UID: "2fcd3188-40ca-4863-b0cb-12780020c1f4"). InnerVolumeSpecName "kube-api-access-2vbsz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:52:51.718414 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.718344 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fcd3188-40ca-4863-b0cb-12780020c1f4-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:52:51.718414 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.718370 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vbsz\" (UniqueName: \"kubernetes.io/projected/2fcd3188-40ca-4863-b0cb-12780020c1f4-kube-api-access-2vbsz\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:52:51.720944 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.720915 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:51.810653 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.810616 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725"] Sep 06 10:52:51.816668 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:51.816636 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-72cb16-b9f649dc-hg725"] Sep 06 10:52:52.928448 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:52.928409 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fcd3188-40ca-4863-b0cb-12780020c1f4" path="/var/lib/kubelet/pods/2fcd3188-40ca-4863-b0cb-12780020c1f4/volumes" Sep 06 10:52:52.929360 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:52.929328 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e528eef8-5955-418b-9967-f30083dc5d40" path="/var/lib/kubelet/pods/e528eef8-5955-418b-9967-f30083dc5d40/volumes" Sep 06 10:52:57.550909 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:57.550866 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:57.651244 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:57.651204 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:57.798768 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:57.798738 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:52:58.215721 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:52:58.215685 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:08.610923 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.610884 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:08.621678 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.621650 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2"] Sep 06 10:53:08.621877 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.621852 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" podUID="975eb1da-1333-42fb-ad66-7cf320a65fd1" containerName="payload-processing" containerID="cri-o://63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585" gracePeriod=30 Sep 06 10:53:08.630422 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.630378 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94"] Sep 06 10:53:08.630955 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.630883 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" podUID="5e4edc92-6408-406d-a71a-80f096fa426a" containerName="payload-pre-processing" containerID="cri-o://534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec" gracePeriod=30 Sep 06 10:53:08.711786 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.711747 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:08.771600 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.771560 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:08.822422 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.822391 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:08.917582 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.917554 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" Sep 06 10:53:08.920417 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.920377 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" Sep 06 10:53:08.978286 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.978252 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc74s\" (UniqueName: \"kubernetes.io/projected/975eb1da-1333-42fb-ad66-7cf320a65fd1-kube-api-access-vc74s\") pod \"975eb1da-1333-42fb-ad66-7cf320a65fd1\" (UID: \"975eb1da-1333-42fb-ad66-7cf320a65fd1\") " Sep 06 10:53:08.978440 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.978310 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e4edc92-6408-406d-a71a-80f096fa426a-plugins-config-volume\") pod \"5e4edc92-6408-406d-a71a-80f096fa426a\" (UID: \"5e4edc92-6408-406d-a71a-80f096fa426a\") " Sep 06 10:53:08.978440 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.978342 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rv6ss\" (UniqueName: \"kubernetes.io/projected/5e4edc92-6408-406d-a71a-80f096fa426a-kube-api-access-rv6ss\") pod \"5e4edc92-6408-406d-a71a-80f096fa426a\" (UID: \"5e4edc92-6408-406d-a71a-80f096fa426a\") " Sep 06 10:53:08.978440 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.978412 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975eb1da-1333-42fb-ad66-7cf320a65fd1-plugins-config-volume\") pod \"975eb1da-1333-42fb-ad66-7cf320a65fd1\" (UID: \"975eb1da-1333-42fb-ad66-7cf320a65fd1\") " Sep 06 10:53:08.978786 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.978684 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e4edc92-6408-406d-a71a-80f096fa426a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5e4edc92-6408-406d-a71a-80f096fa426a" (UID: "5e4edc92-6408-406d-a71a-80f096fa426a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:53:08.978878 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.978835 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/975eb1da-1333-42fb-ad66-7cf320a65fd1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "975eb1da-1333-42fb-ad66-7cf320a65fd1" (UID: "975eb1da-1333-42fb-ad66-7cf320a65fd1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:53:08.980475 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.980456 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/975eb1da-1333-42fb-ad66-7cf320a65fd1-kube-api-access-vc74s" (OuterVolumeSpecName: "kube-api-access-vc74s") pod "975eb1da-1333-42fb-ad66-7cf320a65fd1" (UID: "975eb1da-1333-42fb-ad66-7cf320a65fd1"). InnerVolumeSpecName "kube-api-access-vc74s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:53:08.980545 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:08.980479 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e4edc92-6408-406d-a71a-80f096fa426a-kube-api-access-rv6ss" (OuterVolumeSpecName: "kube-api-access-rv6ss") pod "5e4edc92-6408-406d-a71a-80f096fa426a" (UID: "5e4edc92-6408-406d-a71a-80f096fa426a"). InnerVolumeSpecName "kube-api-access-rv6ss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:53:09.080118 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.080086 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/975eb1da-1333-42fb-ad66-7cf320a65fd1-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:53:09.080118 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.080115 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc74s\" (UniqueName: \"kubernetes.io/projected/975eb1da-1333-42fb-ad66-7cf320a65fd1-kube-api-access-vc74s\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:53:09.080118 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.080147 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e4edc92-6408-406d-a71a-80f096fa426a-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:53:09.080353 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.080159 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rv6ss\" (UniqueName: \"kubernetes.io/projected/5e4edc92-6408-406d-a71a-80f096fa426a-kube-api-access-rv6ss\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:53:09.129594 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.129561 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:09.597453 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.597417 2574 generic.go:356] "Generic (PLEG): container finished" podID="975eb1da-1333-42fb-ad66-7cf320a65fd1" containerID="63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585" exitCode=0 Sep 06 10:53:09.597638 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.597460 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" Sep 06 10:53:09.597638 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.597480 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" event={"ID":"975eb1da-1333-42fb-ad66-7cf320a65fd1","Type":"ContainerDied","Data":"63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585"} Sep 06 10:53:09.597638 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.597534 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2" event={"ID":"975eb1da-1333-42fb-ad66-7cf320a65fd1","Type":"ContainerDied","Data":"7ff0e9c25bfb82905c6ba2b31e28e16693d04d70f9c8d4e28d85c201471bdf7d"} Sep 06 10:53:09.597638 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.597559 2574 scope.go:117] "RemoveContainer" containerID="63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585" Sep 06 10:53:09.598994 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.598970 2574 generic.go:356] "Generic (PLEG): container finished" podID="5e4edc92-6408-406d-a71a-80f096fa426a" containerID="534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec" exitCode=0 Sep 06 10:53:09.599110 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.599005 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" event={"ID":"5e4edc92-6408-406d-a71a-80f096fa426a","Type":"ContainerDied","Data":"534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec"} Sep 06 10:53:09.599110 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.599019 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" Sep 06 10:53:09.599110 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.599058 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94" event={"ID":"5e4edc92-6408-406d-a71a-80f096fa426a","Type":"ContainerDied","Data":"9e86026326aa181024b87a5449d2cf5786e0cf94f67a0699fbb77c041576301e"} Sep 06 10:53:09.608948 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.608926 2574 scope.go:117] "RemoveContainer" containerID="63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585" Sep 06 10:53:09.609485 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:53:09.609456 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585\": container with ID starting with 63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585 not found: ID does not exist" containerID="63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585" Sep 06 10:53:09.609641 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.609611 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585"} err="failed to get container status \"63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585\": rpc error: code = NotFound desc = could not find container \"63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585\": container with ID starting with 63001e3c9abae9a38e2db126cc4b783da6b4814c8b6167083f754608a808c585 not found: ID does not exist" Sep 06 10:53:09.609745 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.609734 2574 scope.go:117] "RemoveContainer" containerID="534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec" Sep 06 10:53:09.619764 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.619745 2574 scope.go:117] "RemoveContainer" containerID="534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec" Sep 06 10:53:09.620029 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:53:09.620011 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec\": container with ID starting with 534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec not found: ID does not exist" containerID="534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec" Sep 06 10:53:09.620081 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.620037 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec"} err="failed to get container status \"534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec\": rpc error: code = NotFound desc = could not find container \"534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec\": container with ID starting with 534befda68dc5feb2a95ecb064839002704eb92050b5205f9c096d247127f5ec not found: ID does not exist" Sep 06 10:53:09.627074 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.627052 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2"] Sep 06 10:53:09.629463 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.629442 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-fbf820-546fc575cf-gmts2"] Sep 06 10:53:09.638540 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.638521 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94"] Sep 06 10:53:09.642293 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:09.642273 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fbf820-7f46996b-xsr94"] Sep 06 10:53:10.926688 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:10.926656 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e4edc92-6408-406d-a71a-80f096fa426a" path="/var/lib/kubelet/pods/5e4edc92-6408-406d-a71a-80f096fa426a/volumes" Sep 06 10:53:10.927113 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:10.927016 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="975eb1da-1333-42fb-ad66-7cf320a65fd1" path="/var/lib/kubelet/pods/975eb1da-1333-42fb-ad66-7cf320a65fd1/volumes" Sep 06 10:53:14.905791 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:14.905740 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:15.006517 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:15.006481 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:15.172233 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:15.172201 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:17.907070 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:17.907034 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:31.934614 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:31.934552 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:32.854360 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:32.854319 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:33.279929 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:33.279893 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:33.562573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:33.562475 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:33.933717 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:33.933684 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:36.925512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:36.925473 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:37.026182 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:37.026151 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:38.162800 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.162772 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:38.263246 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.263194 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:38.435914 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.435878 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:38.619368 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.619336 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:38.719486 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.719396 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:38.725143 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725102 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls"] Sep 06 10:53:38.725514 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725499 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fcd3188-40ca-4863-b0cb-12780020c1f4" containerName="payload-processing" Sep 06 10:53:38.725564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725515 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fcd3188-40ca-4863-b0cb-12780020c1f4" containerName="payload-processing" Sep 06 10:53:38.725564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725524 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e528eef8-5955-418b-9967-f30083dc5d40" containerName="revoke-keys" Sep 06 10:53:38.725564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725530 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e528eef8-5955-418b-9967-f30083dc5d40" containerName="revoke-keys" Sep 06 10:53:38.725564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725540 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e4edc92-6408-406d-a71a-80f096fa426a" containerName="payload-pre-processing" Sep 06 10:53:38.725564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725547 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e4edc92-6408-406d-a71a-80f096fa426a" containerName="payload-pre-processing" Sep 06 10:53:38.725564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725555 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="975eb1da-1333-42fb-ad66-7cf320a65fd1" containerName="payload-processing" Sep 06 10:53:38.725564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725561 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="975eb1da-1333-42fb-ad66-7cf320a65fd1" containerName="payload-processing" Sep 06 10:53:38.725808 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725618 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e4edc92-6408-406d-a71a-80f096fa426a" containerName="payload-pre-processing" Sep 06 10:53:38.725808 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725629 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fcd3188-40ca-4863-b0cb-12780020c1f4" containerName="payload-processing" Sep 06 10:53:38.725808 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725637 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="975eb1da-1333-42fb-ad66-7cf320a65fd1" containerName="payload-processing" Sep 06 10:53:38.725808 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.725643 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e528eef8-5955-418b-9967-f30083dc5d40" containerName="revoke-keys" Sep 06 10:53:38.728620 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.728605 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" Sep 06 10:53:38.730773 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.730751 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-69ef21\"" Sep 06 10:53:38.730868 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.730750 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-69ef21-dockercfg-tlqvd\"" Sep 06 10:53:38.739627 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.739602 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls"] Sep 06 10:53:38.851967 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.851931 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chh6v\" (UniqueName: \"kubernetes.io/projected/13d790b3-4a75-4d1a-9e10-6d10f3ae8206-kube-api-access-chh6v\") pod \"payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls\" (UID: \"13d790b3-4a75-4d1a-9e10-6d10f3ae8206\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" Sep 06 10:53:38.852145 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.852073 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d790b3-4a75-4d1a-9e10-6d10f3ae8206-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls\" (UID: \"13d790b3-4a75-4d1a-9e10-6d10f3ae8206\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" Sep 06 10:53:38.953096 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.953059 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d790b3-4a75-4d1a-9e10-6d10f3ae8206-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls\" (UID: \"13d790b3-4a75-4d1a-9e10-6d10f3ae8206\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" Sep 06 10:53:38.953296 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.953110 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chh6v\" (UniqueName: \"kubernetes.io/projected/13d790b3-4a75-4d1a-9e10-6d10f3ae8206-kube-api-access-chh6v\") pod \"payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls\" (UID: \"13d790b3-4a75-4d1a-9e10-6d10f3ae8206\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" Sep 06 10:53:38.953813 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.953794 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d790b3-4a75-4d1a-9e10-6d10f3ae8206-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls\" (UID: \"13d790b3-4a75-4d1a-9e10-6d10f3ae8206\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" Sep 06 10:53:38.960221 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:38.960190 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chh6v\" (UniqueName: \"kubernetes.io/projected/13d790b3-4a75-4d1a-9e10-6d10f3ae8206-kube-api-access-chh6v\") pod \"payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls\" (UID: \"13d790b3-4a75-4d1a-9e10-6d10f3ae8206\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" Sep 06 10:53:39.040359 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:39.040267 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" Sep 06 10:53:39.310803 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:39.310774 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls"] Sep 06 10:53:39.312103 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:53:39.312078 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d790b3_4a75_4d1a_9e10_6d10f3ae8206.slice/crio-1e908eb1fcb857a3f9b5835689903c984f26dfb4c5c8c0f9de179f4233bddae7 WatchSource:0}: Error finding container 1e908eb1fcb857a3f9b5835689903c984f26dfb4c5c8c0f9de179f4233bddae7: Status 404 returned error can't find the container with id 1e908eb1fcb857a3f9b5835689903c984f26dfb4c5c8c0f9de179f4233bddae7 Sep 06 10:53:39.776338 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:39.776304 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" event={"ID":"13d790b3-4a75-4d1a-9e10-6d10f3ae8206","Type":"ContainerStarted","Data":"b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03"} Sep 06 10:53:39.776512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:39.776362 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" event={"ID":"13d790b3-4a75-4d1a-9e10-6d10f3ae8206","Type":"ContainerStarted","Data":"1e908eb1fcb857a3f9b5835689903c984f26dfb4c5c8c0f9de179f4233bddae7"} Sep 06 10:53:39.776512 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:39.776385 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" Sep 06 10:53:39.792561 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:39.792510 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" podStartSLOduration=1.7924953160000001 podStartE2EDuration="1.792495316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:53:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:53:39.790167178 +0000 UTC m=+1413.372656809" watchObservedRunningTime="2026-09-06 10:53:39.792495316 +0000 UTC m=+1413.374984946" Sep 06 10:53:44.842248 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:44.842213 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:50.785304 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:50.785270 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" Sep 06 10:53:58.477224 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:58.477192 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:53:58.989527 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:53:58.989490 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:01.921028 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:01.920989 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:02.598401 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:02.598367 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:02.900989 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:02.900911 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:02.996330 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:02.996295 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:06.119225 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:06.119190 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:06.207605 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:06.207551 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:07.277177 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.277144 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:07.375526 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.375492 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:07.574589 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.574557 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:07.750952 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.750922 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:07.841698 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.841616 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:07.849378 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.849352 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2"] Sep 06 10:54:07.853012 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.852967 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" Sep 06 10:54:07.855404 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.855379 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-8c14da-dockercfg-9xvch\"" Sep 06 10:54:07.855526 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.855470 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-8c14da\"" Sep 06 10:54:07.861795 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.861775 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2"] Sep 06 10:54:07.886394 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.886362 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x"] Sep 06 10:54:07.889690 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.889673 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" Sep 06 10:54:07.900541 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.900516 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x"] Sep 06 10:54:07.922459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.922431 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwd95\" (UniqueName: \"kubernetes.io/projected/bda25e8c-8cf8-4ab4-890a-93d65599c9aa-kube-api-access-hwd95\") pod \"payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2\" (UID: \"bda25e8c-8cf8-4ab4-890a-93d65599c9aa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" Sep 06 10:54:07.922555 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:07.922463 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda25e8c-8cf8-4ab4-890a-93d65599c9aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2\" (UID: \"bda25e8c-8cf8-4ab4-890a-93d65599c9aa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" Sep 06 10:54:08.023867 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.023828 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwd95\" (UniqueName: \"kubernetes.io/projected/bda25e8c-8cf8-4ab4-890a-93d65599c9aa-kube-api-access-hwd95\") pod \"payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2\" (UID: \"bda25e8c-8cf8-4ab4-890a-93d65599c9aa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" Sep 06 10:54:08.024052 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.023877 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda25e8c-8cf8-4ab4-890a-93d65599c9aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2\" (UID: \"bda25e8c-8cf8-4ab4-890a-93d65599c9aa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" Sep 06 10:54:08.024052 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.023932 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8310840e-b0ae-4ad8-996e-542e4815d33f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x\" (UID: \"8310840e-b0ae-4ad8-996e-542e4815d33f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" Sep 06 10:54:08.024052 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.024014 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf67n\" (UniqueName: \"kubernetes.io/projected/8310840e-b0ae-4ad8-996e-542e4815d33f-kube-api-access-kf67n\") pod \"payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x\" (UID: \"8310840e-b0ae-4ad8-996e-542e4815d33f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" Sep 06 10:54:08.024598 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.024575 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda25e8c-8cf8-4ab4-890a-93d65599c9aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2\" (UID: \"bda25e8c-8cf8-4ab4-890a-93d65599c9aa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" Sep 06 10:54:08.033760 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.033728 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwd95\" (UniqueName: \"kubernetes.io/projected/bda25e8c-8cf8-4ab4-890a-93d65599c9aa-kube-api-access-hwd95\") pod \"payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2\" (UID: \"bda25e8c-8cf8-4ab4-890a-93d65599c9aa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" Sep 06 10:54:08.125557 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.125488 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8310840e-b0ae-4ad8-996e-542e4815d33f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x\" (UID: \"8310840e-b0ae-4ad8-996e-542e4815d33f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" Sep 06 10:54:08.125557 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.125548 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf67n\" (UniqueName: \"kubernetes.io/projected/8310840e-b0ae-4ad8-996e-542e4815d33f-kube-api-access-kf67n\") pod \"payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x\" (UID: \"8310840e-b0ae-4ad8-996e-542e4815d33f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" Sep 06 10:54:08.126116 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.126096 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8310840e-b0ae-4ad8-996e-542e4815d33f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x\" (UID: \"8310840e-b0ae-4ad8-996e-542e4815d33f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" Sep 06 10:54:08.133189 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.133166 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf67n\" (UniqueName: \"kubernetes.io/projected/8310840e-b0ae-4ad8-996e-542e4815d33f-kube-api-access-kf67n\") pod \"payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x\" (UID: \"8310840e-b0ae-4ad8-996e-542e4815d33f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" Sep 06 10:54:08.164459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.164432 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" Sep 06 10:54:08.199580 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.199545 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" Sep 06 10:54:08.442200 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.442172 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2"] Sep 06 10:54:08.443421 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:54:08.443392 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda25e8c_8cf8_4ab4_890a_93d65599c9aa.slice/crio-ad856fab2139097ffdbfd38a13bf399d3003f5ca1c1eb6d538141787a5cffb2e WatchSource:0}: Error finding container ad856fab2139097ffdbfd38a13bf399d3003f5ca1c1eb6d538141787a5cffb2e: Status 404 returned error can't find the container with id ad856fab2139097ffdbfd38a13bf399d3003f5ca1c1eb6d538141787a5cffb2e Sep 06 10:54:08.491185 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.491158 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x"] Sep 06 10:54:08.497178 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:54:08.497153 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8310840e_b0ae_4ad8_996e_542e4815d33f.slice/crio-2e3a160e529a433637e5d028d9091ebb0099e33bfb756de763c0a1d4e2c3635f WatchSource:0}: Error finding container 2e3a160e529a433637e5d028d9091ebb0099e33bfb756de763c0a1d4e2c3635f: Status 404 returned error can't find the container with id 2e3a160e529a433637e5d028d9091ebb0099e33bfb756de763c0a1d4e2c3635f Sep 06 10:54:08.954336 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.954300 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" event={"ID":"bda25e8c-8cf8-4ab4-890a-93d65599c9aa","Type":"ContainerStarted","Data":"f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f"} Sep 06 10:54:08.954542 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.954351 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" event={"ID":"bda25e8c-8cf8-4ab4-890a-93d65599c9aa","Type":"ContainerStarted","Data":"ad856fab2139097ffdbfd38a13bf399d3003f5ca1c1eb6d538141787a5cffb2e"} Sep 06 10:54:08.954542 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.954408 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" Sep 06 10:54:08.956178 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.956117 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" event={"ID":"8310840e-b0ae-4ad8-996e-542e4815d33f","Type":"ContainerStarted","Data":"fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689"} Sep 06 10:54:08.956291 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.956198 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" Sep 06 10:54:08.956291 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.956223 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" event={"ID":"8310840e-b0ae-4ad8-996e-542e4815d33f","Type":"ContainerStarted","Data":"2e3a160e529a433637e5d028d9091ebb0099e33bfb756de763c0a1d4e2c3635f"} Sep 06 10:54:08.972371 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.972316 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" podStartSLOduration=1.972302988 podStartE2EDuration="1.972302988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:54:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:54:08.970239377 +0000 UTC m=+1442.552729010" watchObservedRunningTime="2026-09-06 10:54:08.972302988 +0000 UTC m=+1442.554792618" Sep 06 10:54:08.985701 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:08.985650 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" podStartSLOduration=1.9856359289999999 podStartE2EDuration="1.985635929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:54:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:54:08.984283098 +0000 UTC m=+1442.566772729" watchObservedRunningTime="2026-09-06 10:54:08.985635929 +0000 UTC m=+1442.568125559" Sep 06 10:54:19.964942 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:19.964910 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" Sep 06 10:54:19.965364 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:19.964996 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" Sep 06 10:54:31.125654 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:31.125619 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:36.102491 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:36.102456 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jdfbt_7d9b8533-457c-43a4-a96e-c42b3ad07119/payload-processing/0.log" Sep 06 10:54:36.230363 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:36.230323 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x_8310840e-b0ae-4ad8-996e-542e4815d33f/payload-processing/0.log" Sep 06 10:54:41.131313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:41.131269 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:41.182992 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:41.182961 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:54:47.045315 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:54:47.045283 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:07.171853 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:07.171819 2574 scope.go:117] "RemoveContainer" containerID="289d632e8a420bee3e7c23de447da2b7c6881ca12499745f68fba5515f7e4171" Sep 06 10:55:11.944827 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:11.944793 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jdfbt_7d9b8533-457c-43a4-a96e-c42b3ad07119/payload-processing/0.log" Sep 06 10:55:12.061055 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:12.061027 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x_8310840e-b0ae-4ad8-996e-542e4815d33f/payload-processing/0.log" Sep 06 10:55:12.325048 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:12.324965 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:12.627676 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:12.627597 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:12.845790 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:12.845759 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:15.488399 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:15.488360 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:15.581648 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:15.581614 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:16.658283 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:16.658247 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:16.757179 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:16.757116 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:16.972831 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:16.972793 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:17.142417 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.142384 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:17.233990 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.233883 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:17.235477 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.235456 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv"] Sep 06 10:55:17.239076 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.239059 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" Sep 06 10:55:17.241415 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.241389 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-676ab2\"" Sep 06 10:55:17.248593 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.248562 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv"] Sep 06 10:55:17.254170 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.254139 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rwzp\" (UniqueName: \"kubernetes.io/projected/023fe943-ec5a-4892-9d6b-4139a31bf2b9-kube-api-access-2rwzp\") pod \"payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv\" (UID: \"023fe943-ec5a-4892-9d6b-4139a31bf2b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" Sep 06 10:55:17.254278 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.254263 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/023fe943-ec5a-4892-9d6b-4139a31bf2b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv\" (UID: \"023fe943-ec5a-4892-9d6b-4139a31bf2b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" Sep 06 10:55:17.277622 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.277589 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw"] Sep 06 10:55:17.281079 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.281063 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" Sep 06 10:55:17.286380 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.286359 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-676ab2-dockercfg-6bqk9\"" Sep 06 10:55:17.294521 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.294494 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw"] Sep 06 10:55:17.355921 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.355882 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07471102-ac67-4dab-826b-d8f6b6e3de6f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw\" (UID: \"07471102-ac67-4dab-826b-d8f6b6e3de6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" Sep 06 10:55:17.356261 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.356230 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/023fe943-ec5a-4892-9d6b-4139a31bf2b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv\" (UID: \"023fe943-ec5a-4892-9d6b-4139a31bf2b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" Sep 06 10:55:17.356382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.356312 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q452h\" (UniqueName: \"kubernetes.io/projected/07471102-ac67-4dab-826b-d8f6b6e3de6f-kube-api-access-q452h\") pod \"payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw\" (UID: \"07471102-ac67-4dab-826b-d8f6b6e3de6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" Sep 06 10:55:17.356382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.356358 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rwzp\" (UniqueName: \"kubernetes.io/projected/023fe943-ec5a-4892-9d6b-4139a31bf2b9-kube-api-access-2rwzp\") pod \"payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv\" (UID: \"023fe943-ec5a-4892-9d6b-4139a31bf2b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" Sep 06 10:55:17.357201 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.357180 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/023fe943-ec5a-4892-9d6b-4139a31bf2b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv\" (UID: \"023fe943-ec5a-4892-9d6b-4139a31bf2b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" Sep 06 10:55:17.364584 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.364552 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rwzp\" (UniqueName: \"kubernetes.io/projected/023fe943-ec5a-4892-9d6b-4139a31bf2b9-kube-api-access-2rwzp\") pod \"payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv\" (UID: \"023fe943-ec5a-4892-9d6b-4139a31bf2b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" Sep 06 10:55:17.456926 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.456887 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07471102-ac67-4dab-826b-d8f6b6e3de6f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw\" (UID: \"07471102-ac67-4dab-826b-d8f6b6e3de6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" Sep 06 10:55:17.457098 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.456958 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q452h\" (UniqueName: \"kubernetes.io/projected/07471102-ac67-4dab-826b-d8f6b6e3de6f-kube-api-access-q452h\") pod \"payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw\" (UID: \"07471102-ac67-4dab-826b-d8f6b6e3de6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" Sep 06 10:55:17.457573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.457551 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07471102-ac67-4dab-826b-d8f6b6e3de6f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw\" (UID: \"07471102-ac67-4dab-826b-d8f6b6e3de6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" Sep 06 10:55:17.464101 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.464084 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q452h\" (UniqueName: \"kubernetes.io/projected/07471102-ac67-4dab-826b-d8f6b6e3de6f-kube-api-access-q452h\") pod \"payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw\" (UID: \"07471102-ac67-4dab-826b-d8f6b6e3de6f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" Sep 06 10:55:17.550888 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.550817 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" Sep 06 10:55:17.591372 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.591311 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" Sep 06 10:55:17.826105 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.826077 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv"] Sep 06 10:55:17.827234 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:55:17.827206 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod023fe943_ec5a_4892_9d6b_4139a31bf2b9.slice/crio-3d000e98b593333d3c092a48d7d4e167b9cccfebae0128f90e84ca8e02dfc3e9 WatchSource:0}: Error finding container 3d000e98b593333d3c092a48d7d4e167b9cccfebae0128f90e84ca8e02dfc3e9: Status 404 returned error can't find the container with id 3d000e98b593333d3c092a48d7d4e167b9cccfebae0128f90e84ca8e02dfc3e9 Sep 06 10:55:17.877039 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:17.877016 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw"] Sep 06 10:55:17.877533 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:55:17.877505 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07471102_ac67_4dab_826b_d8f6b6e3de6f.slice/crio-4fef2d2ec5269e811cb7016f27f95ca7ac72644abf5ffad7ead0afe59637de45 WatchSource:0}: Error finding container 4fef2d2ec5269e811cb7016f27f95ca7ac72644abf5ffad7ead0afe59637de45: Status 404 returned error can't find the container with id 4fef2d2ec5269e811cb7016f27f95ca7ac72644abf5ffad7ead0afe59637de45 Sep 06 10:55:18.367320 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:18.367286 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" event={"ID":"023fe943-ec5a-4892-9d6b-4139a31bf2b9","Type":"ContainerStarted","Data":"2312c19763f7db87810a47c8bb04e3a44fa815df831bcd413e9c6d27687b7504"} Sep 06 10:55:18.367537 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:18.367337 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" event={"ID":"023fe943-ec5a-4892-9d6b-4139a31bf2b9","Type":"ContainerStarted","Data":"3d000e98b593333d3c092a48d7d4e167b9cccfebae0128f90e84ca8e02dfc3e9"} Sep 06 10:55:18.367537 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:18.367489 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" Sep 06 10:55:18.369060 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:18.369035 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" event={"ID":"07471102-ac67-4dab-826b-d8f6b6e3de6f","Type":"ContainerStarted","Data":"c3c23a99736873a59681b9aae819d6d3bc3c6dd3f69213cb68c241696909928d"} Sep 06 10:55:18.369224 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:18.369079 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" event={"ID":"07471102-ac67-4dab-826b-d8f6b6e3de6f","Type":"ContainerStarted","Data":"4fef2d2ec5269e811cb7016f27f95ca7ac72644abf5ffad7ead0afe59637de45"} Sep 06 10:55:18.369224 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:18.369105 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" Sep 06 10:55:18.383909 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:18.383867 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" podStartSLOduration=1.383851251 podStartE2EDuration="1.383851251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:55:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:55:18.381412007 +0000 UTC m=+1511.963901643" watchObservedRunningTime="2026-09-06 10:55:18.383851251 +0000 UTC m=+1511.966340882" Sep 06 10:55:18.397944 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:18.397903 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" podStartSLOduration=1.397891088 podStartE2EDuration="1.397891088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:55:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:55:18.394850762 +0000 UTC m=+1511.977340404" watchObservedRunningTime="2026-09-06 10:55:18.397891088 +0000 UTC m=+1511.980380719" Sep 06 10:55:29.378071 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:29.377995 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" Sep 06 10:55:29.378441 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:29.378174 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" Sep 06 10:55:40.320374 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:40.320335 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:40.483588 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:40.483553 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:41.165963 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:41.165930 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k"] Sep 06 10:55:41.169481 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:41.169465 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k" Sep 06 10:55:41.171674 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:41.171651 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 10:55:41.171783 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:41.171731 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\"" Sep 06 10:55:41.176318 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:41.176293 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k"] Sep 06 10:55:41.283715 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:41.283674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e7fdd4-153f-4c19-9893-88cacd5c7369-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k\" (UID: \"79e7fdd4-153f-4c19-9893-88cacd5c7369\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k" Sep 06 10:55:41.384463 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:41.384422 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e7fdd4-153f-4c19-9893-88cacd5c7369-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k\" (UID: \"79e7fdd4-153f-4c19-9893-88cacd5c7369\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k" Sep 06 10:55:41.385048 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:41.385029 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e7fdd4-153f-4c19-9893-88cacd5c7369-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k\" (UID: \"79e7fdd4-153f-4c19-9893-88cacd5c7369\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k" Sep 06 10:55:41.483807 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:41.483727 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k" Sep 06 10:55:41.755337 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:41.755311 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k"] Sep 06 10:55:41.756551 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:55:41.756529 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e7fdd4_153f_4c19_9893_88cacd5c7369.slice/crio-e8658d71d28a68c9e89af526a5f52ea0ecebc4218f1de0d42428ae36e293fc23 WatchSource:0}: Error finding container e8658d71d28a68c9e89af526a5f52ea0ecebc4218f1de0d42428ae36e293fc23: Status 404 returned error can't find the container with id e8658d71d28a68c9e89af526a5f52ea0ecebc4218f1de0d42428ae36e293fc23 Sep 06 10:55:42.519245 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:42.519208 2574 generic.go:356] "Generic (PLEG): container finished" podID="79e7fdd4-153f-4c19-9893-88cacd5c7369" containerID="7b4203b815785337224149920f4790c244f7acbfd21f7e416e81d2a9f7973dff" exitCode=0 Sep 06 10:55:42.519635 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:42.519296 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k" event={"ID":"79e7fdd4-153f-4c19-9893-88cacd5c7369","Type":"ContainerDied","Data":"7b4203b815785337224149920f4790c244f7acbfd21f7e416e81d2a9f7973dff"} Sep 06 10:55:42.519635 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:42.519368 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k" event={"ID":"79e7fdd4-153f-4c19-9893-88cacd5c7369","Type":"ContainerStarted","Data":"e8658d71d28a68c9e89af526a5f52ea0ecebc4218f1de0d42428ae36e293fc23"} Sep 06 10:55:43.658054 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:43.658028 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k" Sep 06 10:55:43.809703 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:43.809600 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e7fdd4-153f-4c19-9893-88cacd5c7369-maas-api-service-ca\") pod \"79e7fdd4-153f-4c19-9893-88cacd5c7369\" (UID: \"79e7fdd4-153f-4c19-9893-88cacd5c7369\") " Sep 06 10:55:43.809976 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:43.809950 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79e7fdd4-153f-4c19-9893-88cacd5c7369-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "79e7fdd4-153f-4c19-9893-88cacd5c7369" (UID: "79e7fdd4-153f-4c19-9893-88cacd5c7369"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:55:43.911069 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:43.911036 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e7fdd4-153f-4c19-9893-88cacd5c7369-maas-api-service-ca\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:55:44.532792 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:44.532764 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k" Sep 06 10:55:44.532959 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:44.532799 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k" event={"ID":"79e7fdd4-153f-4c19-9893-88cacd5c7369","Type":"ContainerDied","Data":"e8658d71d28a68c9e89af526a5f52ea0ecebc4218f1de0d42428ae36e293fc23"} Sep 06 10:55:44.532959 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:44.532847 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8658d71d28a68c9e89af526a5f52ea0ecebc4218f1de0d42428ae36e293fc23" Sep 06 10:55:47.252782 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.252749 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:47.273664 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.273620 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw"] Sep 06 10:55:47.273894 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.273866 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" podUID="07471102-ac67-4dab-826b-d8f6b6e3de6f" containerName="payload-processing" containerID="cri-o://c3c23a99736873a59681b9aae819d6d3bc3c6dd3f69213cb68c241696909928d" gracePeriod=30 Sep 06 10:55:47.285749 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.285716 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv"] Sep 06 10:55:47.285997 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.285964 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" podUID="023fe943-ec5a-4892-9d6b-4139a31bf2b9" containerName="payload-pre-processing" containerID="cri-o://2312c19763f7db87810a47c8bb04e3a44fa815df831bcd413e9c6d27687b7504" gracePeriod=30 Sep 06 10:55:47.313603 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.313571 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:47.373047 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.373005 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:47.428041 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.427986 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k"] Sep 06 10:55:47.431634 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.431567 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:47.431634 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.431604 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-676ab2-h9m7k"] Sep 06 10:55:47.489707 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.489673 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:47.553218 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.553141 2574 generic.go:356] "Generic (PLEG): container finished" podID="023fe943-ec5a-4892-9d6b-4139a31bf2b9" containerID="2312c19763f7db87810a47c8bb04e3a44fa815df831bcd413e9c6d27687b7504" exitCode=0 Sep 06 10:55:47.553218 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.553169 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" event={"ID":"023fe943-ec5a-4892-9d6b-4139a31bf2b9","Type":"ContainerDied","Data":"2312c19763f7db87810a47c8bb04e3a44fa815df831bcd413e9c6d27687b7504"} Sep 06 10:55:47.555117 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.555092 2574 generic.go:356] "Generic (PLEG): container finished" podID="07471102-ac67-4dab-826b-d8f6b6e3de6f" containerID="c3c23a99736873a59681b9aae819d6d3bc3c6dd3f69213cb68c241696909928d" exitCode=0 Sep 06 10:55:47.555242 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.555169 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" event={"ID":"07471102-ac67-4dab-826b-d8f6b6e3de6f","Type":"ContainerDied","Data":"c3c23a99736873a59681b9aae819d6d3bc3c6dd3f69213cb68c241696909928d"} Sep 06 10:55:47.578636 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.578616 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" Sep 06 10:55:47.581755 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.581737 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" Sep 06 10:55:47.745416 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.745380 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rwzp\" (UniqueName: \"kubernetes.io/projected/023fe943-ec5a-4892-9d6b-4139a31bf2b9-kube-api-access-2rwzp\") pod \"023fe943-ec5a-4892-9d6b-4139a31bf2b9\" (UID: \"023fe943-ec5a-4892-9d6b-4139a31bf2b9\") " Sep 06 10:55:47.745607 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.745508 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q452h\" (UniqueName: \"kubernetes.io/projected/07471102-ac67-4dab-826b-d8f6b6e3de6f-kube-api-access-q452h\") pod \"07471102-ac67-4dab-826b-d8f6b6e3de6f\" (UID: \"07471102-ac67-4dab-826b-d8f6b6e3de6f\") " Sep 06 10:55:47.745607 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.745536 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/023fe943-ec5a-4892-9d6b-4139a31bf2b9-plugins-config-volume\") pod \"023fe943-ec5a-4892-9d6b-4139a31bf2b9\" (UID: \"023fe943-ec5a-4892-9d6b-4139a31bf2b9\") " Sep 06 10:55:47.745607 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.745580 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07471102-ac67-4dab-826b-d8f6b6e3de6f-plugins-config-volume\") pod \"07471102-ac67-4dab-826b-d8f6b6e3de6f\" (UID: \"07471102-ac67-4dab-826b-d8f6b6e3de6f\") " Sep 06 10:55:47.745965 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.745935 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/023fe943-ec5a-4892-9d6b-4139a31bf2b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "023fe943-ec5a-4892-9d6b-4139a31bf2b9" (UID: "023fe943-ec5a-4892-9d6b-4139a31bf2b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:55:47.746050 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.745958 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07471102-ac67-4dab-826b-d8f6b6e3de6f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "07471102-ac67-4dab-826b-d8f6b6e3de6f" (UID: "07471102-ac67-4dab-826b-d8f6b6e3de6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:55:47.747551 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.747529 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/023fe943-ec5a-4892-9d6b-4139a31bf2b9-kube-api-access-2rwzp" (OuterVolumeSpecName: "kube-api-access-2rwzp") pod "023fe943-ec5a-4892-9d6b-4139a31bf2b9" (UID: "023fe943-ec5a-4892-9d6b-4139a31bf2b9"). InnerVolumeSpecName "kube-api-access-2rwzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:55:47.747689 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.747671 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07471102-ac67-4dab-826b-d8f6b6e3de6f-kube-api-access-q452h" (OuterVolumeSpecName: "kube-api-access-q452h") pod "07471102-ac67-4dab-826b-d8f6b6e3de6f" (UID: "07471102-ac67-4dab-826b-d8f6b6e3de6f"). InnerVolumeSpecName "kube-api-access-q452h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:55:47.834675 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.834543 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:47.846496 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.846475 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rwzp\" (UniqueName: \"kubernetes.io/projected/023fe943-ec5a-4892-9d6b-4139a31bf2b9-kube-api-access-2rwzp\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:55:47.846496 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.846497 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q452h\" (UniqueName: \"kubernetes.io/projected/07471102-ac67-4dab-826b-d8f6b6e3de6f-kube-api-access-q452h\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:55:47.846639 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.846506 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/023fe943-ec5a-4892-9d6b-4139a31bf2b9-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:55:47.846639 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:47.846515 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07471102-ac67-4dab-826b-d8f6b6e3de6f-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:55:48.563655 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.563623 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" event={"ID":"023fe943-ec5a-4892-9d6b-4139a31bf2b9","Type":"ContainerDied","Data":"3d000e98b593333d3c092a48d7d4e167b9cccfebae0128f90e84ca8e02dfc3e9"} Sep 06 10:55:48.564071 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.563689 2574 scope.go:117] "RemoveContainer" containerID="2312c19763f7db87810a47c8bb04e3a44fa815df831bcd413e9c6d27687b7504" Sep 06 10:55:48.564071 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.563638 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv" Sep 06 10:55:48.565696 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.565678 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" Sep 06 10:55:48.565818 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.565784 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw" event={"ID":"07471102-ac67-4dab-826b-d8f6b6e3de6f","Type":"ContainerDied","Data":"4fef2d2ec5269e811cb7016f27f95ca7ac72644abf5ffad7ead0afe59637de45"} Sep 06 10:55:48.576597 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.576048 2574 scope.go:117] "RemoveContainer" containerID="c3c23a99736873a59681b9aae819d6d3bc3c6dd3f69213cb68c241696909928d" Sep 06 10:55:48.591842 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.591814 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw"] Sep 06 10:55:48.594484 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.594459 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-676ab2-55879bdf67-6qtzw"] Sep 06 10:55:48.604762 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.604695 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv"] Sep 06 10:55:48.608236 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.608212 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-676ab2-787944759d-9mnlv"] Sep 06 10:55:48.927194 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.927161 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="023fe943-ec5a-4892-9d6b-4139a31bf2b9" path="/var/lib/kubelet/pods/023fe943-ec5a-4892-9d6b-4139a31bf2b9/volumes" Sep 06 10:55:48.927573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.927560 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07471102-ac67-4dab-826b-d8f6b6e3de6f" path="/var/lib/kubelet/pods/07471102-ac67-4dab-826b-d8f6b6e3de6f/volumes" Sep 06 10:55:48.927851 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:48.927840 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79e7fdd4-153f-4c19-9893-88cacd5c7369" path="/var/lib/kubelet/pods/79e7fdd4-153f-4c19-9893-88cacd5c7369/volumes" Sep 06 10:55:53.465640 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:53.465610 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:53.564840 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:53.564794 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:53.724821 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:53.724714 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:57.242510 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.242472 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7"] Sep 06 10:55:57.243063 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.243042 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e7fdd4-153f-4c19-9893-88cacd5c7369" containerName="revoke-keys" Sep 06 10:55:57.243178 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.243067 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e7fdd4-153f-4c19-9893-88cacd5c7369" containerName="revoke-keys" Sep 06 10:55:57.243178 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.243088 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="023fe943-ec5a-4892-9d6b-4139a31bf2b9" containerName="payload-pre-processing" Sep 06 10:55:57.243178 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.243098 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="023fe943-ec5a-4892-9d6b-4139a31bf2b9" containerName="payload-pre-processing" Sep 06 10:55:57.243178 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.243142 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07471102-ac67-4dab-826b-d8f6b6e3de6f" containerName="payload-processing" Sep 06 10:55:57.243178 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.243152 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="07471102-ac67-4dab-826b-d8f6b6e3de6f" containerName="payload-processing" Sep 06 10:55:57.243439 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.243271 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="07471102-ac67-4dab-826b-d8f6b6e3de6f" containerName="payload-processing" Sep 06 10:55:57.243439 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.243285 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="023fe943-ec5a-4892-9d6b-4139a31bf2b9" containerName="payload-pre-processing" Sep 06 10:55:57.243439 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.243296 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="79e7fdd4-153f-4c19-9893-88cacd5c7369" containerName="revoke-keys" Sep 06 10:55:57.248046 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.248026 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7" Sep 06 10:55:57.251197 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.251171 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\"" Sep 06 10:55:57.251456 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.251436 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 10:55:57.252853 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.252827 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7"] Sep 06 10:55:57.331191 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.331154 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7\" (UID: \"e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7" Sep 06 10:55:57.348727 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.348673 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:55:57.432383 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.432347 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7\" (UID: \"e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7" Sep 06 10:55:57.432987 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.432964 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7\" (UID: \"e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7" Sep 06 10:55:57.561330 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.561224 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7" Sep 06 10:55:57.832469 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:57.832441 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7"] Sep 06 10:55:57.833804 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:55:57.833776 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode17b8e1f_08fb_4be5_bb79_a5a2ca9e1c47.slice/crio-00e12aa3f6a02ed59a1e79b0192447582a6ff81f0800c24e069659e065798084 WatchSource:0}: Error finding container 00e12aa3f6a02ed59a1e79b0192447582a6ff81f0800c24e069659e065798084: Status 404 returned error can't find the container with id 00e12aa3f6a02ed59a1e79b0192447582a6ff81f0800c24e069659e065798084 Sep 06 10:55:58.634052 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:58.634021 2574 generic.go:356] "Generic (PLEG): container finished" podID="e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47" containerID="4afbd5d0771b455bb7851e78f81a3e9b5abc1a74a8c94a20df81f891b737f683" exitCode=0 Sep 06 10:55:58.634459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:58.634078 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7" event={"ID":"e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47","Type":"ContainerDied","Data":"4afbd5d0771b455bb7851e78f81a3e9b5abc1a74a8c94a20df81f891b737f683"} Sep 06 10:55:58.634459 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:58.634115 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7" event={"ID":"e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47","Type":"ContainerStarted","Data":"00e12aa3f6a02ed59a1e79b0192447582a6ff81f0800c24e069659e065798084"} Sep 06 10:55:59.778337 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:59.778314 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7" Sep 06 10:55:59.854607 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:59.854571 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47-maas-api-service-ca\") pod \"e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47\" (UID: \"e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47\") " Sep 06 10:55:59.854984 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:59.854957 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47" (UID: "e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:55:59.955915 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:55:59.955873 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47-maas-api-service-ca\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:56:00.646726 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:00.646690 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7" Sep 06 10:56:00.646904 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:00.646696 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7" event={"ID":"e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47","Type":"ContainerDied","Data":"00e12aa3f6a02ed59a1e79b0192447582a6ff81f0800c24e069659e065798084"} Sep 06 10:56:00.646904 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:00.646817 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00e12aa3f6a02ed59a1e79b0192447582a6ff81f0800c24e069659e065798084" Sep 06 10:56:12.683194 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:12.683156 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:56:12.711969 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:12.711935 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls"] Sep 06 10:56:12.712215 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:12.712188 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" podUID="13d790b3-4a75-4d1a-9e10-6d10f3ae8206" containerName="payload-pre-processing" containerID="cri-o://b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03" gracePeriod=30 Sep 06 10:56:12.784012 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:12.783980 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:56:12.862909 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:12.862875 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:56:12.863234 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:12.863217 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7"] Sep 06 10:56:12.865739 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:12.865714 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-69ef21-f9dm7"] Sep 06 10:56:12.915608 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:12.915581 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:56:12.927543 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:12.927512 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47" path="/var/lib/kubelet/pods/e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47/volumes" Sep 06 10:56:12.991401 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:12.991377 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" Sep 06 10:56:13.074325 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.074296 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d790b3-4a75-4d1a-9e10-6d10f3ae8206-plugins-config-volume\") pod \"13d790b3-4a75-4d1a-9e10-6d10f3ae8206\" (UID: \"13d790b3-4a75-4d1a-9e10-6d10f3ae8206\") " Sep 06 10:56:13.074510 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.074497 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chh6v\" (UniqueName: \"kubernetes.io/projected/13d790b3-4a75-4d1a-9e10-6d10f3ae8206-kube-api-access-chh6v\") pod \"13d790b3-4a75-4d1a-9e10-6d10f3ae8206\" (UID: \"13d790b3-4a75-4d1a-9e10-6d10f3ae8206\") " Sep 06 10:56:13.074794 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.074757 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13d790b3-4a75-4d1a-9e10-6d10f3ae8206-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "13d790b3-4a75-4d1a-9e10-6d10f3ae8206" (UID: "13d790b3-4a75-4d1a-9e10-6d10f3ae8206"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:56:13.076375 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.076342 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d790b3-4a75-4d1a-9e10-6d10f3ae8206-kube-api-access-chh6v" (OuterVolumeSpecName: "kube-api-access-chh6v") pod "13d790b3-4a75-4d1a-9e10-6d10f3ae8206" (UID: "13d790b3-4a75-4d1a-9e10-6d10f3ae8206"). InnerVolumeSpecName "kube-api-access-chh6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:56:13.175864 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.175829 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d790b3-4a75-4d1a-9e10-6d10f3ae8206-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:56:13.175864 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.175856 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chh6v\" (UniqueName: \"kubernetes.io/projected/13d790b3-4a75-4d1a-9e10-6d10f3ae8206-kube-api-access-chh6v\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:56:13.230895 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.230807 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:56:13.728480 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.728447 2574 generic.go:356] "Generic (PLEG): container finished" podID="13d790b3-4a75-4d1a-9e10-6d10f3ae8206" containerID="b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03" exitCode=0 Sep 06 10:56:13.728944 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.728493 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" Sep 06 10:56:13.728944 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.728532 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" event={"ID":"13d790b3-4a75-4d1a-9e10-6d10f3ae8206","Type":"ContainerDied","Data":"b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03"} Sep 06 10:56:13.728944 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.728583 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls" event={"ID":"13d790b3-4a75-4d1a-9e10-6d10f3ae8206","Type":"ContainerDied","Data":"1e908eb1fcb857a3f9b5835689903c984f26dfb4c5c8c0f9de179f4233bddae7"} Sep 06 10:56:13.728944 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.728613 2574 scope.go:117] "RemoveContainer" containerID="b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03" Sep 06 10:56:13.738065 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.738050 2574 scope.go:117] "RemoveContainer" containerID="b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03" Sep 06 10:56:13.738343 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:56:13.738324 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03\": container with ID starting with b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03 not found: ID does not exist" containerID="b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03" Sep 06 10:56:13.738388 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.738352 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03"} err="failed to get container status \"b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03\": rpc error: code = NotFound desc = could not find container \"b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03\": container with ID starting with b856f55fcf532f92487aac773d85f6740c609aa07a378c9ced8253d19bda4c03 not found: ID does not exist" Sep 06 10:56:13.749714 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.749693 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls"] Sep 06 10:56:13.754386 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:13.754364 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-69ef21-7d97c7dfd6-smqls"] Sep 06 10:56:14.926829 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:14.926796 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13d790b3-4a75-4d1a-9e10-6d10f3ae8206" path="/var/lib/kubelet/pods/13d790b3-4a75-4d1a-9e10-6d10f3ae8206/volumes" Sep 06 10:56:17.256861 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:17.256823 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:56:18.257437 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:18.257401 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:56:44.669693 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:44.669657 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:56:54.630090 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:54.630054 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:56:55.114434 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:56:55.114398 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:00.742227 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:00.742110 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:06.157488 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.157439 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:06.258460 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.258429 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:06.429462 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.429432 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:06.780987 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.780908 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz"] Sep 06 10:57:06.781329 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.781316 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13d790b3-4a75-4d1a-9e10-6d10f3ae8206" containerName="payload-pre-processing" Sep 06 10:57:06.781382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.781331 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="13d790b3-4a75-4d1a-9e10-6d10f3ae8206" containerName="payload-pre-processing" Sep 06 10:57:06.781382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.781351 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47" containerName="revoke-keys" Sep 06 10:57:06.781382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.781357 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47" containerName="revoke-keys" Sep 06 10:57:06.781494 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.781433 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e17b8e1f-08fb-4be5-bb79-a5a2ca9e1c47" containerName="revoke-keys" Sep 06 10:57:06.781494 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.781440 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="13d790b3-4a75-4d1a-9e10-6d10f3ae8206" containerName="payload-pre-processing" Sep 06 10:57:06.784645 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.784628 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz" Sep 06 10:57:06.786885 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.786868 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\"" Sep 06 10:57:06.786885 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.786879 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 10:57:06.791987 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.791962 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz"] Sep 06 10:57:06.882449 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.882402 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd42fa60-e398-4f18-988c-d85308e3ef3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz\" (UID: \"dd42fa60-e398-4f18-988c-d85308e3ef3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz" Sep 06 10:57:06.983437 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.983409 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd42fa60-e398-4f18-988c-d85308e3ef3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz\" (UID: \"dd42fa60-e398-4f18-988c-d85308e3ef3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz" Sep 06 10:57:06.985654 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.985635 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 06 10:57:06.995052 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:06.995028 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd42fa60-e398-4f18-988c-d85308e3ef3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz\" (UID: \"dd42fa60-e398-4f18-988c-d85308e3ef3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz" Sep 06 10:57:07.041714 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:07.041629 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:07.099674 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:07.099645 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\"" Sep 06 10:57:07.106613 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:07.106568 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz" Sep 06 10:57:07.383238 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:07.383206 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz"] Sep 06 10:57:07.384677 ip-10-0-139-139 kubenswrapper[2574]: W0906 10:57:07.384651 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd42fa60_e398_4f18_988c_d85308e3ef3e.slice/crio-096699d2282d0c8dfecb870ea23415114f8871343475c98afbfd9d2733f6d06e WatchSource:0}: Error finding container 096699d2282d0c8dfecb870ea23415114f8871343475c98afbfd9d2733f6d06e: Status 404 returned error can't find the container with id 096699d2282d0c8dfecb870ea23415114f8871343475c98afbfd9d2733f6d06e Sep 06 10:57:08.038423 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:08.038392 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz" event={"ID":"dd42fa60-e398-4f18-988c-d85308e3ef3e","Type":"ContainerStarted","Data":"cfe1a23505eb65f9f9031166f6e0b5c48c221974d1ca710eb772a16b53a5834d"} Sep 06 10:57:08.038593 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:08.038442 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz" event={"ID":"dd42fa60-e398-4f18-988c-d85308e3ef3e","Type":"ContainerStarted","Data":"096699d2282d0c8dfecb870ea23415114f8871343475c98afbfd9d2733f6d06e"} Sep 06 10:57:08.052633 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:08.052589 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz" podStartSLOduration=2.05257533 podStartE2EDuration="2.05257533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 10:57:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 10:57:08.051998706 +0000 UTC m=+1621.634488339" watchObservedRunningTime="2026-09-06 10:57:08.05257533 +0000 UTC m=+1621.635064961" Sep 06 10:57:09.045757 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:09.045715 2574 generic.go:356] "Generic (PLEG): container finished" podID="dd42fa60-e398-4f18-988c-d85308e3ef3e" containerID="cfe1a23505eb65f9f9031166f6e0b5c48c221974d1ca710eb772a16b53a5834d" exitCode=0 Sep 06 10:57:09.046160 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:09.045810 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz" event={"ID":"dd42fa60-e398-4f18-988c-d85308e3ef3e","Type":"ContainerDied","Data":"cfe1a23505eb65f9f9031166f6e0b5c48c221974d1ca710eb772a16b53a5834d"} Sep 06 10:57:10.200218 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:10.200190 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz" Sep 06 10:57:10.315716 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:10.315623 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd42fa60-e398-4f18-988c-d85308e3ef3e-maas-api-service-ca\") pod \"dd42fa60-e398-4f18-988c-d85308e3ef3e\" (UID: \"dd42fa60-e398-4f18-988c-d85308e3ef3e\") " Sep 06 10:57:10.315983 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:10.315953 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd42fa60-e398-4f18-988c-d85308e3ef3e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dd42fa60-e398-4f18-988c-d85308e3ef3e" (UID: "dd42fa60-e398-4f18-988c-d85308e3ef3e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:57:10.417064 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:10.417030 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd42fa60-e398-4f18-988c-d85308e3ef3e-maas-api-service-ca\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:11.058657 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:11.058627 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz" Sep 06 10:57:11.058657 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:11.058639 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz" event={"ID":"dd42fa60-e398-4f18-988c-d85308e3ef3e","Type":"ContainerDied","Data":"096699d2282d0c8dfecb870ea23415114f8871343475c98afbfd9d2733f6d06e"} Sep 06 10:57:11.058846 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:11.058683 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="096699d2282d0c8dfecb870ea23415114f8871343475c98afbfd9d2733f6d06e" Sep 06 10:57:17.231510 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.231457 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:17.261463 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.261429 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x"] Sep 06 10:57:17.261696 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.261671 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" podUID="8310840e-b0ae-4ad8-996e-542e4815d33f" containerName="payload-processing" containerID="cri-o://fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689" gracePeriod=30 Sep 06 10:57:17.268551 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.268518 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2"] Sep 06 10:57:17.268774 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.268738 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" podUID="bda25e8c-8cf8-4ab4-890a-93d65599c9aa" containerName="payload-pre-processing" containerID="cri-o://f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f" gracePeriod=30 Sep 06 10:57:17.300050 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.300014 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:17.359696 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.359664 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:17.415114 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.415077 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:17.415415 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.415390 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz"] Sep 06 10:57:17.417867 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.417839 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c14da-67lsz"] Sep 06 10:57:17.475212 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.475118 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:17.562498 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.562474 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" Sep 06 10:57:17.565072 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.565053 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" Sep 06 10:57:17.683770 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.683738 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwd95\" (UniqueName: \"kubernetes.io/projected/bda25e8c-8cf8-4ab4-890a-93d65599c9aa-kube-api-access-hwd95\") pod \"bda25e8c-8cf8-4ab4-890a-93d65599c9aa\" (UID: \"bda25e8c-8cf8-4ab4-890a-93d65599c9aa\") " Sep 06 10:57:17.683770 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.683772 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8310840e-b0ae-4ad8-996e-542e4815d33f-plugins-config-volume\") pod \"8310840e-b0ae-4ad8-996e-542e4815d33f\" (UID: \"8310840e-b0ae-4ad8-996e-542e4815d33f\") " Sep 06 10:57:17.683974 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.683796 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kf67n\" (UniqueName: \"kubernetes.io/projected/8310840e-b0ae-4ad8-996e-542e4815d33f-kube-api-access-kf67n\") pod \"8310840e-b0ae-4ad8-996e-542e4815d33f\" (UID: \"8310840e-b0ae-4ad8-996e-542e4815d33f\") " Sep 06 10:57:17.683974 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.683876 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda25e8c-8cf8-4ab4-890a-93d65599c9aa-plugins-config-volume\") pod \"bda25e8c-8cf8-4ab4-890a-93d65599c9aa\" (UID: \"bda25e8c-8cf8-4ab4-890a-93d65599c9aa\") " Sep 06 10:57:17.684177 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.684114 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8310840e-b0ae-4ad8-996e-542e4815d33f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8310840e-b0ae-4ad8-996e-542e4815d33f" (UID: "8310840e-b0ae-4ad8-996e-542e4815d33f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:57:17.684312 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.684289 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8310840e-b0ae-4ad8-996e-542e4815d33f-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:17.684367 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.684295 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda25e8c-8cf8-4ab4-890a-93d65599c9aa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bda25e8c-8cf8-4ab4-890a-93d65599c9aa" (UID: "bda25e8c-8cf8-4ab4-890a-93d65599c9aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:57:17.685957 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.685931 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8310840e-b0ae-4ad8-996e-542e4815d33f-kube-api-access-kf67n" (OuterVolumeSpecName: "kube-api-access-kf67n") pod "8310840e-b0ae-4ad8-996e-542e4815d33f" (UID: "8310840e-b0ae-4ad8-996e-542e4815d33f"). InnerVolumeSpecName "kube-api-access-kf67n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:57:17.686052 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.686018 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bda25e8c-8cf8-4ab4-890a-93d65599c9aa-kube-api-access-hwd95" (OuterVolumeSpecName: "kube-api-access-hwd95") pod "bda25e8c-8cf8-4ab4-890a-93d65599c9aa" (UID: "bda25e8c-8cf8-4ab4-890a-93d65599c9aa"). InnerVolumeSpecName "kube-api-access-hwd95". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:57:17.785490 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.785456 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwd95\" (UniqueName: \"kubernetes.io/projected/bda25e8c-8cf8-4ab4-890a-93d65599c9aa-kube-api-access-hwd95\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:17.785490 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.785483 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf67n\" (UniqueName: \"kubernetes.io/projected/8310840e-b0ae-4ad8-996e-542e4815d33f-kube-api-access-kf67n\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:17.785490 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.785492 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda25e8c-8cf8-4ab4-890a-93d65599c9aa-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:17.789252 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:17.789222 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:18.101135 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.101093 2574 generic.go:356] "Generic (PLEG): container finished" podID="bda25e8c-8cf8-4ab4-890a-93d65599c9aa" containerID="f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f" exitCode=0 Sep 06 10:57:18.101298 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.101156 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" Sep 06 10:57:18.101298 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.101171 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" event={"ID":"bda25e8c-8cf8-4ab4-890a-93d65599c9aa","Type":"ContainerDied","Data":"f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f"} Sep 06 10:57:18.101298 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.101225 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2" event={"ID":"bda25e8c-8cf8-4ab4-890a-93d65599c9aa","Type":"ContainerDied","Data":"ad856fab2139097ffdbfd38a13bf399d3003f5ca1c1eb6d538141787a5cffb2e"} Sep 06 10:57:18.101298 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.101248 2574 scope.go:117] "RemoveContainer" containerID="f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f" Sep 06 10:57:18.102646 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.102615 2574 generic.go:356] "Generic (PLEG): container finished" podID="8310840e-b0ae-4ad8-996e-542e4815d33f" containerID="fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689" exitCode=0 Sep 06 10:57:18.102772 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.102665 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" Sep 06 10:57:18.102772 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.102719 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" event={"ID":"8310840e-b0ae-4ad8-996e-542e4815d33f","Type":"ContainerDied","Data":"fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689"} Sep 06 10:57:18.102772 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.102761 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x" event={"ID":"8310840e-b0ae-4ad8-996e-542e4815d33f","Type":"ContainerDied","Data":"2e3a160e529a433637e5d028d9091ebb0099e33bfb756de763c0a1d4e2c3635f"} Sep 06 10:57:18.110669 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.110652 2574 scope.go:117] "RemoveContainer" containerID="f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f" Sep 06 10:57:18.110949 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:57:18.110929 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f\": container with ID starting with f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f not found: ID does not exist" containerID="f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f" Sep 06 10:57:18.111019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.110956 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f"} err="failed to get container status \"f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f\": rpc error: code = NotFound desc = could not find container \"f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f\": container with ID starting with f6a74ed9dd3914a4cd4089c8dbbd7736b12e3c4e46c9e8b12364dd91d12ebb6f not found: ID does not exist" Sep 06 10:57:18.111019 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.110976 2574 scope.go:117] "RemoveContainer" containerID="fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689" Sep 06 10:57:18.119312 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.119294 2574 scope.go:117] "RemoveContainer" containerID="fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689" Sep 06 10:57:18.119588 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:57:18.119570 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689\": container with ID starting with fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689 not found: ID does not exist" containerID="fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689" Sep 06 10:57:18.119639 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.119594 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689"} err="failed to get container status \"fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689\": rpc error: code = NotFound desc = could not find container \"fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689\": container with ID starting with fa6f856ff0d1e6b5164c7c4ea44514547192341cdd0e5643ff0d7bf401d15689 not found: ID does not exist" Sep 06 10:57:18.133184 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.128343 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2"] Sep 06 10:57:18.136441 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.136418 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8c14da-74b6445fd6-682n2"] Sep 06 10:57:18.146256 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.146236 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x"] Sep 06 10:57:18.150573 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.150553 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-8c14da-9f8ddf9b5-g7s6x"] Sep 06 10:57:18.926118 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.926081 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8310840e-b0ae-4ad8-996e-542e4815d33f" path="/var/lib/kubelet/pods/8310840e-b0ae-4ad8-996e-542e4815d33f/volumes" Sep 06 10:57:18.926482 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.926472 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bda25e8c-8cf8-4ab4-890a-93d65599c9aa" path="/var/lib/kubelet/pods/bda25e8c-8cf8-4ab4-890a-93d65599c9aa/volumes" Sep 06 10:57:18.926766 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:18.926753 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd42fa60-e398-4f18-988c-d85308e3ef3e" path="/var/lib/kubelet/pods/dd42fa60-e398-4f18-988c-d85308e3ef3e/volumes" Sep 06 10:57:23.634727 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:23.634680 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:23.735779 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:23.735747 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:23.896726 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:23.896634 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:24.345340 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:24.345304 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:34.662432 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:34.662390 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:34.683300 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:34.683265 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r"] Sep 06 10:57:34.683593 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:34.683563 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" podUID="ce904edc-a4bc-4738-aa3c-80de703e9790" containerName="payload-processing" containerID="cri-o://a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f" gracePeriod=30 Sep 06 10:57:34.762184 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:34.762150 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:34.832548 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:34.832515 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:34.891714 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:34.891684 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:34.956382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:34.956357 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" Sep 06 10:57:35.045033 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.045003 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjk4d\" (UniqueName: \"kubernetes.io/projected/ce904edc-a4bc-4738-aa3c-80de703e9790-kube-api-access-vjk4d\") pod \"ce904edc-a4bc-4738-aa3c-80de703e9790\" (UID: \"ce904edc-a4bc-4738-aa3c-80de703e9790\") " Sep 06 10:57:35.045226 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.045202 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce904edc-a4bc-4738-aa3c-80de703e9790-plugins-config-volume\") pod \"ce904edc-a4bc-4738-aa3c-80de703e9790\" (UID: \"ce904edc-a4bc-4738-aa3c-80de703e9790\") " Sep 06 10:57:35.045589 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.045557 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce904edc-a4bc-4738-aa3c-80de703e9790-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce904edc-a4bc-4738-aa3c-80de703e9790" (UID: "ce904edc-a4bc-4738-aa3c-80de703e9790"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:57:35.047189 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.047164 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce904edc-a4bc-4738-aa3c-80de703e9790-kube-api-access-vjk4d" (OuterVolumeSpecName: "kube-api-access-vjk4d") pod "ce904edc-a4bc-4738-aa3c-80de703e9790" (UID: "ce904edc-a4bc-4738-aa3c-80de703e9790"). InnerVolumeSpecName "kube-api-access-vjk4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:57:35.146313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.146279 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce904edc-a4bc-4738-aa3c-80de703e9790-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:35.146313 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.146306 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjk4d\" (UniqueName: \"kubernetes.io/projected/ce904edc-a4bc-4738-aa3c-80de703e9790-kube-api-access-vjk4d\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:35.204521 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.204432 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:35.207608 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.207583 2574 generic.go:356] "Generic (PLEG): container finished" podID="ce904edc-a4bc-4738-aa3c-80de703e9790" containerID="a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f" exitCode=0 Sep 06 10:57:35.207742 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.207629 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" Sep 06 10:57:35.207742 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.207667 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" event={"ID":"ce904edc-a4bc-4738-aa3c-80de703e9790","Type":"ContainerDied","Data":"a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f"} Sep 06 10:57:35.207742 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.207725 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r" event={"ID":"ce904edc-a4bc-4738-aa3c-80de703e9790","Type":"ContainerDied","Data":"bf5cda2e5ff6174891998a699338a85d372595bf95740f5ff9bfe9c7f0fac5ba"} Sep 06 10:57:35.207901 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.207755 2574 scope.go:117] "RemoveContainer" containerID="a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f" Sep 06 10:57:35.216731 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.216712 2574 scope.go:117] "RemoveContainer" containerID="a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f" Sep 06 10:57:35.216987 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:57:35.216970 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f\": container with ID starting with a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f not found: ID does not exist" containerID="a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f" Sep 06 10:57:35.217039 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.216996 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f"} err="failed to get container status \"a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f\": rpc error: code = NotFound desc = could not find container \"a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f\": container with ID starting with a0f77a9caf44ec66ba86f3054c2e13c4694fffcbe07ed0d9009ac283d5c6818f not found: ID does not exist" Sep 06 10:57:35.228361 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.228342 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r"] Sep 06 10:57:35.233982 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:35.233963 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2eb161-74458547d9-lp42r"] Sep 06 10:57:36.926588 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:36.926548 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce904edc-a4bc-4738-aa3c-80de703e9790" path="/var/lib/kubelet/pods/ce904edc-a4bc-4738-aa3c-80de703e9790/volumes" Sep 06 10:57:41.086302 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:41.086263 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:41.141168 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:41.141115 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:41.319033 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:41.318999 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5"] Sep 06 10:57:41.319249 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:41.319216 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" containerID="cri-o://a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384" gracePeriod=30 Sep 06 10:57:41.339800 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:41.339723 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:41.857250 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:41.857215 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:45.355424 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:45.355386 2574 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 06 10:57:45.355796 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:45.355446 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 06 10:57:46.575360 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.575331 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:57:46.656957 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.656856 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-credential-socket\") pod \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " Sep 06 10:57:46.656957 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.656897 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-data\") pod \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " Sep 06 10:57:46.656957 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.656922 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istiod-ca-cert\") pod \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " Sep 06 10:57:46.656957 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.656960 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-token\") pod \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " Sep 06 10:57:46.657349 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657005 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-workload-certs\") pod \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " Sep 06 10:57:46.657349 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657026 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-envoy\") pod \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " Sep 06 10:57:46.657349 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657103 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-workload-socket\") pod \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " Sep 06 10:57:46.657349 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657184 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnlvx\" (UniqueName: \"kubernetes.io/projected/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-kube-api-access-tnlvx\") pod \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " Sep 06 10:57:46.657349 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657235 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-podinfo\") pod \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\" (UID: \"045c7a18-e0f4-4e90-945c-d6512aa7d1f4\") " Sep 06 10:57:46.657606 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657228 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "045c7a18-e0f4-4e90-945c-d6512aa7d1f4" (UID: "045c7a18-e0f4-4e90-945c-d6512aa7d1f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:57:46.657606 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657245 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-data" (OuterVolumeSpecName: "istio-data") pod "045c7a18-e0f4-4e90-945c-d6512aa7d1f4" (UID: "045c7a18-e0f4-4e90-945c-d6512aa7d1f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:57:46.657606 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657372 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "045c7a18-e0f4-4e90-945c-d6512aa7d1f4" (UID: "045c7a18-e0f4-4e90-945c-d6512aa7d1f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:57:46.657606 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657431 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "045c7a18-e0f4-4e90-945c-d6512aa7d1f4" (UID: "045c7a18-e0f4-4e90-945c-d6512aa7d1f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:57:46.657606 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657482 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "045c7a18-e0f4-4e90-945c-d6512aa7d1f4" (UID: "045c7a18-e0f4-4e90-945c-d6512aa7d1f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:57:46.657841 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657651 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-credential-socket\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:46.657841 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657666 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-data\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:46.657841 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657680 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istiod-ca-cert\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:46.657841 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657689 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-workload-certs\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:46.657841 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.657696 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-workload-socket\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:46.659337 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.659313 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "045c7a18-e0f4-4e90-945c-d6512aa7d1f4" (UID: "045c7a18-e0f4-4e90-945c-d6512aa7d1f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 10:57:46.659474 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.659378 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "045c7a18-e0f4-4e90-945c-d6512aa7d1f4" (UID: "045c7a18-e0f4-4e90-945c-d6512aa7d1f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 06 10:57:46.659613 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.659595 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-token" (OuterVolumeSpecName: "istio-token") pod "045c7a18-e0f4-4e90-945c-d6512aa7d1f4" (UID: "045c7a18-e0f4-4e90-945c-d6512aa7d1f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:57:46.659688 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.659653 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-kube-api-access-tnlvx" (OuterVolumeSpecName: "kube-api-access-tnlvx") pod "045c7a18-e0f4-4e90-945c-d6512aa7d1f4" (UID: "045c7a18-e0f4-4e90-945c-d6512aa7d1f4"). InnerVolumeSpecName "kube-api-access-tnlvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:57:46.758333 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.758297 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-podinfo\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:46.758333 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.758329 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-token\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:46.758333 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.758341 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-istio-envoy\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:46.758561 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:46.758354 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnlvx\" (UniqueName: \"kubernetes.io/projected/045c7a18-e0f4-4e90-945c-d6512aa7d1f4-kube-api-access-tnlvx\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:47.278378 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:47.278279 2574 generic.go:356] "Generic (PLEG): container finished" podID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerID="a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384" exitCode=0 Sep 06 10:57:47.278378 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:47.278328 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" Sep 06 10:57:47.278378 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:47.278356 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" event={"ID":"045c7a18-e0f4-4e90-945c-d6512aa7d1f4","Type":"ContainerDied","Data":"a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384"} Sep 06 10:57:47.278663 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:47.278411 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5" event={"ID":"045c7a18-e0f4-4e90-945c-d6512aa7d1f4","Type":"ContainerDied","Data":"5e6d02651cd6fc20e78418340fca2fdfc3ff2156f2488f878c5c8fb81727dd74"} Sep 06 10:57:47.278663 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:47.278433 2574 scope.go:117] "RemoveContainer" containerID="a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384" Sep 06 10:57:47.287629 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:47.287607 2574 scope.go:117] "RemoveContainer" containerID="e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e" Sep 06 10:57:47.295367 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:47.295341 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5"] Sep 06 10:57:47.296333 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:47.296316 2574 scope.go:117] "RemoveContainer" containerID="a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384" Sep 06 10:57:47.296744 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:57:47.296709 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384\": container with ID starting with a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384 not found: ID does not exist" containerID="a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384" Sep 06 10:57:47.296826 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:47.296747 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384"} err="failed to get container status \"a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384\": rpc error: code = NotFound desc = could not find container \"a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384\": container with ID starting with a4f3a23b06bfdedb25a2451b3eaed6df1b3720d102ae6003eb4934b54ca7b384 not found: ID does not exist" Sep 06 10:57:47.296826 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:47.296771 2574 scope.go:117] "RemoveContainer" containerID="e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e" Sep 06 10:57:47.297093 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:57:47.297074 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e\": container with ID starting with e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e not found: ID does not exist" containerID="e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e" Sep 06 10:57:47.297184 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:47.297099 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e"} err="failed to get container status \"e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e\": rpc error: code = NotFound desc = could not find container \"e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e\": container with ID starting with e25d7f73a0c7b5d657829ab70afd1a081ffb98b83fc5c98036a101946b58221e not found: ID does not exist" Sep 06 10:57:47.298869 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:47.298850 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2eb161-openshift-default-5b675dc6b6-4vqc5"] Sep 06 10:57:48.933673 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:48.933628 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" path="/var/lib/kubelet/pods/045c7a18-e0f4-4e90-945c-d6512aa7d1f4/volumes" Sep 06 10:57:52.233895 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.233863 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:52.263116 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.263084 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv"] Sep 06 10:57:52.263370 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.263341 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" podUID="48b48797-0260-4391-9fe6-ab7eaa3b73c7" containerName="payload-processing" containerID="cri-o://e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c" gracePeriod=30 Sep 06 10:57:52.269249 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.269216 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2"] Sep 06 10:57:52.269518 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.269464 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" podUID="090b8b25-589c-41d0-8bda-6df732167703" containerName="payload-pre-processing" containerID="cri-o://7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308" gracePeriod=30 Sep 06 10:57:52.301778 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.301742 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:52.355382 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.355356 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:52.419115 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.419079 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:52.483264 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.483225 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:52.536050 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.536031 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" Sep 06 10:57:52.538832 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.538813 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" Sep 06 10:57:52.713977 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.713950 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gqg8\" (UniqueName: \"kubernetes.io/projected/090b8b25-589c-41d0-8bda-6df732167703-kube-api-access-9gqg8\") pod \"090b8b25-589c-41d0-8bda-6df732167703\" (UID: \"090b8b25-589c-41d0-8bda-6df732167703\") " Sep 06 10:57:52.714199 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.713988 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsmc9\" (UniqueName: \"kubernetes.io/projected/48b48797-0260-4391-9fe6-ab7eaa3b73c7-kube-api-access-gsmc9\") pod \"48b48797-0260-4391-9fe6-ab7eaa3b73c7\" (UID: \"48b48797-0260-4391-9fe6-ab7eaa3b73c7\") " Sep 06 10:57:52.714199 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.714070 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48b48797-0260-4391-9fe6-ab7eaa3b73c7-plugins-config-volume\") pod \"48b48797-0260-4391-9fe6-ab7eaa3b73c7\" (UID: \"48b48797-0260-4391-9fe6-ab7eaa3b73c7\") " Sep 06 10:57:52.714199 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.714085 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/090b8b25-589c-41d0-8bda-6df732167703-plugins-config-volume\") pod \"090b8b25-589c-41d0-8bda-6df732167703\" (UID: \"090b8b25-589c-41d0-8bda-6df732167703\") " Sep 06 10:57:52.714518 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.714487 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48b48797-0260-4391-9fe6-ab7eaa3b73c7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "48b48797-0260-4391-9fe6-ab7eaa3b73c7" (UID: "48b48797-0260-4391-9fe6-ab7eaa3b73c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:57:52.714611 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.714498 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/090b8b25-589c-41d0-8bda-6df732167703-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "090b8b25-589c-41d0-8bda-6df732167703" (UID: "090b8b25-589c-41d0-8bda-6df732167703"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 06 10:57:52.716165 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.716137 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48b48797-0260-4391-9fe6-ab7eaa3b73c7-kube-api-access-gsmc9" (OuterVolumeSpecName: "kube-api-access-gsmc9") pod "48b48797-0260-4391-9fe6-ab7eaa3b73c7" (UID: "48b48797-0260-4391-9fe6-ab7eaa3b73c7"). InnerVolumeSpecName "kube-api-access-gsmc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:57:52.716273 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.716148 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/090b8b25-589c-41d0-8bda-6df732167703-kube-api-access-9gqg8" (OuterVolumeSpecName: "kube-api-access-9gqg8") pod "090b8b25-589c-41d0-8bda-6df732167703" (UID: "090b8b25-589c-41d0-8bda-6df732167703"). InnerVolumeSpecName "kube-api-access-9gqg8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 10:57:52.789391 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.789316 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:52.815418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.815392 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48b48797-0260-4391-9fe6-ab7eaa3b73c7-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:52.815418 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.815416 2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/090b8b25-589c-41d0-8bda-6df732167703-plugins-config-volume\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:52.815564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.815425 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gqg8\" (UniqueName: \"kubernetes.io/projected/090b8b25-589c-41d0-8bda-6df732167703-kube-api-access-9gqg8\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:52.815564 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:52.815434 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsmc9\" (UniqueName: \"kubernetes.io/projected/48b48797-0260-4391-9fe6-ab7eaa3b73c7-kube-api-access-gsmc9\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 10:57:53.316437 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.316398 2574 generic.go:356] "Generic (PLEG): container finished" podID="48b48797-0260-4391-9fe6-ab7eaa3b73c7" containerID="e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c" exitCode=0 Sep 06 10:57:53.316831 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.316444 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" Sep 06 10:57:53.316831 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.316453 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" event={"ID":"48b48797-0260-4391-9fe6-ab7eaa3b73c7","Type":"ContainerDied","Data":"e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c"} Sep 06 10:57:53.316831 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.316517 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv" event={"ID":"48b48797-0260-4391-9fe6-ab7eaa3b73c7","Type":"ContainerDied","Data":"c8f63a18e46dc83d35220f87526f35ce50e6b84e5c30c6dfc7e336917a8918c0"} Sep 06 10:57:53.316831 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.316548 2574 scope.go:117] "RemoveContainer" containerID="e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c" Sep 06 10:57:53.318097 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.318073 2574 generic.go:356] "Generic (PLEG): container finished" podID="090b8b25-589c-41d0-8bda-6df732167703" containerID="7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308" exitCode=0 Sep 06 10:57:53.318238 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.318143 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" Sep 06 10:57:53.318238 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.318118 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" event={"ID":"090b8b25-589c-41d0-8bda-6df732167703","Type":"ContainerDied","Data":"7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308"} Sep 06 10:57:53.318345 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.318273 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2" event={"ID":"090b8b25-589c-41d0-8bda-6df732167703","Type":"ContainerDied","Data":"d150f036e519d26e42a13b33bc52e0f3e89a4c1c201d2a03c77c57a8b85e088b"} Sep 06 10:57:53.325882 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.325866 2574 scope.go:117] "RemoveContainer" containerID="e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c" Sep 06 10:57:53.326186 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:57:53.326165 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c\": container with ID starting with e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c not found: ID does not exist" containerID="e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c" Sep 06 10:57:53.326264 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.326191 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c"} err="failed to get container status \"e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c\": rpc error: code = NotFound desc = could not find container \"e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c\": container with ID starting with e30a5bc759e8f2d53bc243203a8d317b98c321bec5092ab1548da65690bfe39c not found: ID does not exist" Sep 06 10:57:53.326264 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.326222 2574 scope.go:117] "RemoveContainer" containerID="7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308" Sep 06 10:57:53.334352 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.334327 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2"] Sep 06 10:57:53.334614 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.334589 2574 scope.go:117] "RemoveContainer" containerID="7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308" Sep 06 10:57:53.335098 ip-10-0-139-139 kubenswrapper[2574]: E0906 10:57:53.335047 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308\": container with ID starting with 7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308 not found: ID does not exist" containerID="7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308" Sep 06 10:57:53.335230 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.335093 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308"} err="failed to get container status \"7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308\": rpc error: code = NotFound desc = could not find container \"7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308\": container with ID starting with 7ff77996be2dd23f772beeec60d59fc6efabf78bb2a7d866a03b83523d912308 not found: ID does not exist" Sep 06 10:57:53.337371 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.337354 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-eb4b45-6c5867c78-mr7w2"] Sep 06 10:57:53.346509 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.346489 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv"] Sep 06 10:57:53.348482 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:53.348465 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-eb4b45-8495c6d5b5-78dkv"] Sep 06 10:57:54.925948 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:54.925915 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="090b8b25-589c-41d0-8bda-6df732167703" path="/var/lib/kubelet/pods/090b8b25-589c-41d0-8bda-6df732167703/volumes" Sep 06 10:57:54.926351 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:54.926314 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48b48797-0260-4391-9fe6-ab7eaa3b73c7" path="/var/lib/kubelet/pods/48b48797-0260-4391-9fe6-ab7eaa3b73c7/volumes" Sep 06 10:57:58.755802 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:58.755767 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:58.813764 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:58.813731 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:57:59.013626 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:57:59.013549 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:58:07.232868 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:58:07.232832 2574 scope.go:117] "RemoveContainer" containerID="87cdb85a611419149bc3550ed96e57348bdba0b5b3b6a6e87db2bede8fc8e5ac" Sep 06 10:58:28.530533 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:58:28.530497 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:59:07.275946 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:59:07.275912 2574 scope.go:117] "RemoveContainer" containerID="c350935930ac11ca1d8a96bfbd424c51ce46412be1f109e251541d3cecc6ea7e" Sep 06 10:59:07.284341 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:59:07.284321 2574 scope.go:117] "RemoveContainer" containerID="0fde5f9d646c1ae648caa0bf1ff418ae044be0e7d282c4a22de93f410f8b7b2a" Sep 06 10:59:22.909644 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:59:22.909596 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:59:24.925896 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:59:24.925857 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 10:59:26.491113 ip-10-0-139-139 kubenswrapper[2574]: I0906 10:59:26.491074 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 11:00:00.128868 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.128785 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811540-m82pp"] Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129137 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="090b8b25-589c-41d0-8bda-6df732167703" containerName="payload-pre-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129150 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="090b8b25-589c-41d0-8bda-6df732167703" containerName="payload-pre-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129165 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129172 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129182 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce904edc-a4bc-4738-aa3c-80de703e9790" containerName="payload-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129189 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce904edc-a4bc-4738-aa3c-80de703e9790" containerName="payload-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129201 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48b48797-0260-4391-9fe6-ab7eaa3b73c7" containerName="payload-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129207 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="48b48797-0260-4391-9fe6-ab7eaa3b73c7" containerName="payload-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129213 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bda25e8c-8cf8-4ab4-890a-93d65599c9aa" containerName="payload-pre-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129218 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bda25e8c-8cf8-4ab4-890a-93d65599c9aa" containerName="payload-pre-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129229 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8310840e-b0ae-4ad8-996e-542e4815d33f" containerName="payload-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129234 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8310840e-b0ae-4ad8-996e-542e4815d33f" containerName="payload-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129244 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd42fa60-e398-4f18-988c-d85308e3ef3e" containerName="revoke-keys" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129250 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd42fa60-e398-4f18-988c-d85308e3ef3e" containerName="revoke-keys" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129304 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8310840e-b0ae-4ad8-996e-542e4815d33f" containerName="payload-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129312 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="090b8b25-589c-41d0-8bda-6df732167703" containerName="payload-pre-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129319 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce904edc-a4bc-4738-aa3c-80de703e9790" containerName="payload-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129326 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129331 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bda25e8c-8cf8-4ab4-890a-93d65599c9aa" containerName="payload-pre-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129336 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd42fa60-e398-4f18-988c-d85308e3ef3e" containerName="revoke-keys" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129342 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="48b48797-0260-4391-9fe6-ab7eaa3b73c7" containerName="payload-processing" Sep 06 11:00:00.129358 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.129348 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" Sep 06 11:00:00.132336 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.132319 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811540-m82pp" Sep 06 11:00:00.134494 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.134472 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppq2n\"" Sep 06 11:00:00.138752 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.138641 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811540-m82pp"] Sep 06 11:00:00.143898 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.143854 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811540-m82pp" Sep 06 11:00:00.418278 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:00.418255 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811540-m82pp"] Sep 06 11:00:00.420613 ip-10-0-139-139 kubenswrapper[2574]: W0906 11:00:00.420581 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7e5ae8_97de_434a_9f4d_dbfa5c012e21.slice/crio-eaed41a512254200038d04ecad97a1ab7277dadde70ef1b57ea0c7dba702bbbc WatchSource:0}: Error finding container eaed41a512254200038d04ecad97a1ab7277dadde70ef1b57ea0c7dba702bbbc: Status 404 returned error can't find the container with id eaed41a512254200038d04ecad97a1ab7277dadde70ef1b57ea0c7dba702bbbc Sep 06 11:00:01.027706 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:01.027611 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811540-m82pp" event={"ID":"3e7e5ae8-97de-434a-9f4d-dbfa5c012e21","Type":"ContainerStarted","Data":"4fa88d291f56eacf20fc5a016098f6d93a20c250935cc89f1c25c077c7be370e"} Sep 06 11:00:01.027706 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:01.027675 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811540-m82pp" event={"ID":"3e7e5ae8-97de-434a-9f4d-dbfa5c012e21","Type":"ContainerStarted","Data":"eaed41a512254200038d04ecad97a1ab7277dadde70ef1b57ea0c7dba702bbbc"} Sep 06 11:00:01.042333 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:01.042283 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811540-m82pp" podStartSLOduration=1.042268404 podStartE2EDuration="1.042268404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 11:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 11:00:01.040279817 +0000 UTC m=+1794.622769448" watchObservedRunningTime="2026-09-06 11:00:01.042268404 +0000 UTC m=+1794.624758032" Sep 06 11:00:02.034378 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:02.034345 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e7e5ae8-97de-434a-9f4d-dbfa5c012e21" containerID="4fa88d291f56eacf20fc5a016098f6d93a20c250935cc89f1c25c077c7be370e" exitCode=0 Sep 06 11:00:02.034765 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:02.034390 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811540-m82pp" event={"ID":"3e7e5ae8-97de-434a-9f4d-dbfa5c012e21","Type":"ContainerDied","Data":"4fa88d291f56eacf20fc5a016098f6d93a20c250935cc89f1c25c077c7be370e"} Sep 06 11:00:03.166187 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:03.166164 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811540-m82pp" Sep 06 11:00:04.046536 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:04.046503 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811540-m82pp" Sep 06 11:00:04.046716 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:04.046503 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811540-m82pp" event={"ID":"3e7e5ae8-97de-434a-9f4d-dbfa5c012e21","Type":"ContainerDied","Data":"eaed41a512254200038d04ecad97a1ab7277dadde70ef1b57ea0c7dba702bbbc"} Sep 06 11:00:04.046716 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:00:04.046630 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eaed41a512254200038d04ecad97a1ab7277dadde70ef1b57ea0c7dba702bbbc" Sep 06 11:02:07.323264 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:07.323231 2574 scope.go:117] "RemoveContainer" containerID="4afbd5d0771b455bb7851e78f81a3e9b5abc1a74a8c94a20df81f891b737f683" Sep 06 11:02:07.331737 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:07.331714 2574 scope.go:117] "RemoveContainer" containerID="7b4203b815785337224149920f4790c244f7acbfd21f7e416e81d2a9f7973dff" Sep 06 11:02:17.479740 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:17.479696 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 11:02:20.224323 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.224280 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c"] Sep 06 11:02:20.224687 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.224652 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" Sep 06 11:02:20.224687 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.224664 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="045c7a18-e0f4-4e90-945c-d6512aa7d1f4" containerName="istio-proxy" Sep 06 11:02:20.224687 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.224684 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e7e5ae8-97de-434a-9f4d-dbfa5c012e21" containerName="cleanup" Sep 06 11:02:20.224793 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.224690 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e7e5ae8-97de-434a-9f4d-dbfa5c012e21" containerName="cleanup" Sep 06 11:02:20.224793 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.224751 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e7e5ae8-97de-434a-9f4d-dbfa5c012e21" containerName="cleanup" Sep 06 11:02:20.227768 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.227740 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.229909 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.229886 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-6eb51a07-kserve-self-signed-certs\"" Sep 06 11:02:20.236900 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.236876 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c"] Sep 06 11:02:20.250266 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.250233 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 11:02:20.280119 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.280085 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-model-cache\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.280283 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.280144 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25584cbd-05fa-47d3-9dc1-2f187329a39f-tls-certs\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.280283 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.280230 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-home\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.280362 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.280285 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shs6s\" (UniqueName: \"kubernetes.io/projected/25584cbd-05fa-47d3-9dc1-2f187329a39f-kube-api-access-shs6s\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.280362 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.280319 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-dshm\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.280428 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.280388 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-tmp-dir\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.381327 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.381288 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-home\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.381507 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.381336 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shs6s\" (UniqueName: \"kubernetes.io/projected/25584cbd-05fa-47d3-9dc1-2f187329a39f-kube-api-access-shs6s\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.381507 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.381353 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-dshm\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.381507 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.381483 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-tmp-dir\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.381680 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.381535 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-model-cache\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.381680 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.381563 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25584cbd-05fa-47d3-9dc1-2f187329a39f-tls-certs\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.381816 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.381792 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-home\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.381923 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.381851 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-tmp-dir\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.381923 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.381864 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-model-cache\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.383568 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.383547 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-dshm\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.383934 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.383914 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25584cbd-05fa-47d3-9dc1-2f187329a39f-tls-certs\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.395472 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.395446 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shs6s\" (UniqueName: \"kubernetes.io/projected/25584cbd-05fa-47d3-9dc1-2f187329a39f-kube-api-access-shs6s\") pod \"e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.539458 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.539341 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:20.814565 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:20.814477 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c"] Sep 06 11:02:20.817259 ip-10-0-139-139 kubenswrapper[2574]: W0906 11:02:20.817236 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25584cbd_05fa_47d3_9dc1_2f187329a39f.slice/crio-295ebd97faca0b3795c4c14c752ea4a73c9163b66b82b514bcbcab5d364c3622 WatchSource:0}: Error finding container 295ebd97faca0b3795c4c14c752ea4a73c9163b66b82b514bcbcab5d364c3622: Status 404 returned error can't find the container with id 295ebd97faca0b3795c4c14c752ea4a73c9163b66b82b514bcbcab5d364c3622 Sep 06 11:02:21.802510 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:21.802474 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" event={"ID":"25584cbd-05fa-47d3-9dc1-2f187329a39f","Type":"ContainerStarted","Data":"80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207"} Sep 06 11:02:21.802914 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:21.802519 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" event={"ID":"25584cbd-05fa-47d3-9dc1-2f187329a39f","Type":"ContainerStarted","Data":"295ebd97faca0b3795c4c14c752ea4a73c9163b66b82b514bcbcab5d364c3622"} Sep 06 11:02:21.817847 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:21.817796 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" podStartSLOduration=1.817781496 podStartE2EDuration="1.817781496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 11:02:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 11:02:21.816294521 +0000 UTC m=+1935.398784144" watchObservedRunningTime="2026-09-06 11:02:21.817781496 +0000 UTC m=+1935.400271124" Sep 06 11:02:30.540373 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:30.540319 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:30.540373 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:30.540389 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:30.552730 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:30.552703 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:30.862399 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:30.862312 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:37.536708 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:37.536668 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 11:02:39.316218 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:39.316184 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c"] Sep 06 11:02:39.332645 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:39.332599 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 11:02:54.336231 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.336187 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" podUID="25584cbd-05fa-47d3-9dc1-2f187329a39f" containerName="main" containerID="cri-o://80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207" gracePeriod=45 Sep 06 11:02:54.575071 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.575047 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:54.606311 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.606230 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25584cbd-05fa-47d3-9dc1-2f187329a39f-tls-certs\") pod \"25584cbd-05fa-47d3-9dc1-2f187329a39f\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " Sep 06 11:02:54.606311 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.606293 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-tmp-dir\") pod \"25584cbd-05fa-47d3-9dc1-2f187329a39f\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " Sep 06 11:02:54.606529 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.606317 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shs6s\" (UniqueName: \"kubernetes.io/projected/25584cbd-05fa-47d3-9dc1-2f187329a39f-kube-api-access-shs6s\") pod \"25584cbd-05fa-47d3-9dc1-2f187329a39f\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " Sep 06 11:02:54.606529 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.606368 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-home\") pod \"25584cbd-05fa-47d3-9dc1-2f187329a39f\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " Sep 06 11:02:54.606529 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.606384 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-model-cache\") pod \"25584cbd-05fa-47d3-9dc1-2f187329a39f\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " Sep 06 11:02:54.606529 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.606424 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-dshm\") pod \"25584cbd-05fa-47d3-9dc1-2f187329a39f\" (UID: \"25584cbd-05fa-47d3-9dc1-2f187329a39f\") " Sep 06 11:02:54.606757 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.606526 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "25584cbd-05fa-47d3-9dc1-2f187329a39f" (UID: "25584cbd-05fa-47d3-9dc1-2f187329a39f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 11:02:54.606757 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.606632 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-home" (OuterVolumeSpecName: "home") pod "25584cbd-05fa-47d3-9dc1-2f187329a39f" (UID: "25584cbd-05fa-47d3-9dc1-2f187329a39f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 11:02:54.606757 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.606728 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-model-cache" (OuterVolumeSpecName: "model-cache") pod "25584cbd-05fa-47d3-9dc1-2f187329a39f" (UID: "25584cbd-05fa-47d3-9dc1-2f187329a39f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 11:02:54.606926 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.606769 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-tmp-dir\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 11:02:54.606926 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.606790 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-home\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 11:02:54.608539 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.608504 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25584cbd-05fa-47d3-9dc1-2f187329a39f-kube-api-access-shs6s" (OuterVolumeSpecName: "kube-api-access-shs6s") pod "25584cbd-05fa-47d3-9dc1-2f187329a39f" (UID: "25584cbd-05fa-47d3-9dc1-2f187329a39f"). InnerVolumeSpecName "kube-api-access-shs6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 06 11:02:54.608751 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.608733 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-dshm" (OuterVolumeSpecName: "dshm") pod "25584cbd-05fa-47d3-9dc1-2f187329a39f" (UID: "25584cbd-05fa-47d3-9dc1-2f187329a39f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 06 11:02:54.608751 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.608740 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25584cbd-05fa-47d3-9dc1-2f187329a39f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "25584cbd-05fa-47d3-9dc1-2f187329a39f" (UID: "25584cbd-05fa-47d3-9dc1-2f187329a39f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 06 11:02:54.707361 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.707335 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-model-cache\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 11:02:54.707361 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.707357 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25584cbd-05fa-47d3-9dc1-2f187329a39f-dshm\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 11:02:54.707361 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.707366 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25584cbd-05fa-47d3-9dc1-2f187329a39f-tls-certs\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 11:02:54.707554 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.707374 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shs6s\" (UniqueName: \"kubernetes.io/projected/25584cbd-05fa-47d3-9dc1-2f187329a39f-kube-api-access-shs6s\") on node \"ip-10-0-139-139.ec2.internal\" DevicePath \"\"" Sep 06 11:02:54.978173 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.978115 2574 generic.go:356] "Generic (PLEG): container finished" podID="25584cbd-05fa-47d3-9dc1-2f187329a39f" containerID="80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207" exitCode=0 Sep 06 11:02:54.978364 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.978180 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" Sep 06 11:02:54.978364 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.978201 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" event={"ID":"25584cbd-05fa-47d3-9dc1-2f187329a39f","Type":"ContainerDied","Data":"80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207"} Sep 06 11:02:54.978364 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.978254 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c" event={"ID":"25584cbd-05fa-47d3-9dc1-2f187329a39f","Type":"ContainerDied","Data":"295ebd97faca0b3795c4c14c752ea4a73c9163b66b82b514bcbcab5d364c3622"} Sep 06 11:02:54.978364 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.978276 2574 scope.go:117] "RemoveContainer" containerID="80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207" Sep 06 11:02:54.987284 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.987269 2574 scope.go:117] "RemoveContainer" containerID="80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207" Sep 06 11:02:54.987544 ip-10-0-139-139 kubenswrapper[2574]: E0906 11:02:54.987524 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207\": container with ID starting with 80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207 not found: ID does not exist" containerID="80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207" Sep 06 11:02:54.987595 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.987552 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207"} err="failed to get container status \"80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207\": rpc error: code = NotFound desc = could not find container \"80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207\": container with ID starting with 80776a839a5506b64970aeba7691abb528c69ede3c623aaffba9bba2bfd68207 not found: ID does not exist" Sep 06 11:02:54.994235 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.994212 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c"] Sep 06 11:02:54.996318 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:54.996299 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-6eb51a07-kserve-6479f8847-5ng9c"] Sep 06 11:02:56.932540 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:02:56.932505 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25584cbd-05fa-47d3-9dc1-2f187329a39f" path="/var/lib/kubelet/pods/25584cbd-05fa-47d3-9dc1-2f187329a39f/volumes" Sep 06 11:03:43.026796 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:43.026770 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zn5wj_ea239aef-7690-41a3-8a78-5e216efc0b45/server/0.log" Sep 06 11:03:43.150727 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:43.150695 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tvq92_7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2/manager/0.log" Sep 06 11:03:43.260418 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:43.260388 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645f664568-2zfbs_1d45ffaa-1462-4ec1-a397-9cca1f483cce/copy-manifests/0.log" Sep 06 11:03:43.279506 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:43.279422 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645f664568-2zfbs_1d45ffaa-1462-4ec1-a397-9cca1f483cce/manager/0.log" Sep 06 11:03:44.057612 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:44.057580 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29811525-zzzjn_1fa0dc5d-1798-4c50-a58d-b4db321a42af/cleanup/0.log" Sep 06 11:03:44.167076 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:44.167048 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29811540-m82pp_3e7e5ae8-97de-434a-9f4d-dbfa5c012e21/cleanup/0.log" Sep 06 11:03:46.081659 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:46.081627 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk_2ce36125-a847-43b9-bab2-29bd4bf0f8f5/istio-proxy/0.log" Sep 06 11:03:46.199513 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:46.199473 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-m7ftd_0719fc1d-2dea-450e-b161-8e2361629cf0/discovery/0.log" Sep 06 11:03:46.766327 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:46.766299 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jdfbt_7d9b8533-457c-43a4-a96e-c42b3ad07119/payload-processing/0.log" Sep 06 11:03:47.224682 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:47.224642 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-2dgzg_836e23c7-15f2-4613-a54c-16e02646aadc/main/0.log" Sep 06 11:03:47.555788 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:47.555710 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-92sxc_03d1a85a-49f6-4eb8-9e9f-30cdf8a2e815/main/0.log" Sep 06 11:03:47.899524 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:47.899444 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 11:03:47.999773 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:47.999743 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 11:03:53.177824 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:53.177793 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 11:03:53.279389 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:53.279355 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 11:03:56.015167 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:56.015117 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n424n_d037fa14-5fbf-4635-92df-3b0117254fca/global-pull-secret-syncer/0.log" Sep 06 11:03:56.102854 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:56.102822 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" Sep 06 11:03:56.222090 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:03:56.222054 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-139.ec2.internal_d442c7b86c5fc53cff3e2ee8151b31bc/haproxy/0.log" Sep 06 11:04:03.029485 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.029427 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_42e1d522-2daf-4a64-8f7a-f2f95fa428bb/alertmanager/0.log" Sep 06 11:04:03.069210 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.069183 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_42e1d522-2daf-4a64-8f7a-f2f95fa428bb/config-reloader/0.log" Sep 06 11:04:03.109977 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.109946 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_42e1d522-2daf-4a64-8f7a-f2f95fa428bb/kube-rbac-proxy-web/0.log" Sep 06 11:04:03.149161 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.149117 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_42e1d522-2daf-4a64-8f7a-f2f95fa428bb/kube-rbac-proxy/0.log" Sep 06 11:04:03.189290 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.189260 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_42e1d522-2daf-4a64-8f7a-f2f95fa428bb/kube-rbac-proxy-metric/0.log" Sep 06 11:04:03.234100 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.234070 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_42e1d522-2daf-4a64-8f7a-f2f95fa428bb/prom-label-proxy/0.log" Sep 06 11:04:03.267064 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.267034 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_42e1d522-2daf-4a64-8f7a-f2f95fa428bb/init-config-reloader/0.log" Sep 06 11:04:03.687874 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.687841 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-skwvr_a560b82a-e82b-487d-bd6c-a27417548ae4/node-exporter/0.log" Sep 06 11:04:03.708098 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.708069 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-skwvr_a560b82a-e82b-487d-bd6c-a27417548ae4/kube-rbac-proxy/0.log" Sep 06 11:04:03.729912 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.729887 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-skwvr_a560b82a-e82b-487d-bd6c-a27417548ae4/init-textfile/0.log" Sep 06 11:04:03.762631 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.762605 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-hv5vr_691ff084-48cd-4dce-ad4a-4cc02968306f/kube-rbac-proxy-main/0.log" Sep 06 11:04:03.783446 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.783352 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-hv5vr_691ff084-48cd-4dce-ad4a-4cc02968306f/kube-rbac-proxy-self/0.log" Sep 06 11:04:03.803057 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:03.803022 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-hv5vr_691ff084-48cd-4dce-ad4a-4cc02968306f/openshift-state-metrics/0.log" Sep 06 11:04:04.102508 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:04.102419 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-nsdkx_295fe1ab-b385-4ed7-b35f-f8cc4b9037b5/prometheus-operator-admission-webhook/0.log" Sep 06 11:04:04.209060 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:04.209034 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f855d5774-qgpvp_f21b4bb0-7741-421a-a77c-95b7c59a79a0/thanos-query/0.log" Sep 06 11:04:04.229514 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:04.229484 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f855d5774-qgpvp_f21b4bb0-7741-421a-a77c-95b7c59a79a0/kube-rbac-proxy-web/0.log" Sep 06 11:04:04.251750 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:04.251724 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f855d5774-qgpvp_f21b4bb0-7741-421a-a77c-95b7c59a79a0/kube-rbac-proxy/0.log" Sep 06 11:04:04.277776 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:04.277749 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f855d5774-qgpvp_f21b4bb0-7741-421a-a77c-95b7c59a79a0/prom-label-proxy/0.log" Sep 06 11:04:04.297141 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:04.297095 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f855d5774-qgpvp_f21b4bb0-7741-421a-a77c-95b7c59a79a0/kube-rbac-proxy-rules/0.log" Sep 06 11:04:04.318601 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:04.318574 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f855d5774-qgpvp_f21b4bb0-7741-421a-a77c-95b7c59a79a0/kube-rbac-proxy-metrics/0.log" Sep 06 11:04:06.549779 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.549737 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442"] Sep 06 11:04:06.550457 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.550433 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25584cbd-05fa-47d3-9dc1-2f187329a39f" containerName="main" Sep 06 11:04:06.550457 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.550459 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="25584cbd-05fa-47d3-9dc1-2f187329a39f" containerName="main" Sep 06 11:04:06.550604 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.550545 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="25584cbd-05fa-47d3-9dc1-2f187329a39f" containerName="main" Sep 06 11:04:06.553842 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.553821 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.556074 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.556048 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pw9j6\"/\"kube-root-ca.crt\"" Sep 06 11:04:06.556203 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.556089 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pw9j6\"/\"default-dockercfg-xq2z4\"" Sep 06 11:04:06.556203 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.556186 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pw9j6\"/\"openshift-service-ca.crt\"" Sep 06 11:04:06.561749 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.561727 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442"] Sep 06 11:04:06.659590 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.659553 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ed6e515-cb3a-4258-86cb-a799985b3e4f-podres\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.659785 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.659628 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ed6e515-cb3a-4258-86cb-a799985b3e4f-sys\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.659785 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.659652 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ed6e515-cb3a-4258-86cb-a799985b3e4f-lib-modules\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.659785 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.659674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pjpz\" (UniqueName: \"kubernetes.io/projected/5ed6e515-cb3a-4258-86cb-a799985b3e4f-kube-api-access-8pjpz\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.659785 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.659705 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ed6e515-cb3a-4258-86cb-a799985b3e4f-proc\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.760271 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.760233 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ed6e515-cb3a-4258-86cb-a799985b3e4f-proc\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.760454 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.760294 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ed6e515-cb3a-4258-86cb-a799985b3e4f-podres\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.760454 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.760340 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ed6e515-cb3a-4258-86cb-a799985b3e4f-sys\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.760454 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.760361 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ed6e515-cb3a-4258-86cb-a799985b3e4f-lib-modules\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.760454 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.760381 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pjpz\" (UniqueName: \"kubernetes.io/projected/5ed6e515-cb3a-4258-86cb-a799985b3e4f-kube-api-access-8pjpz\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.760454 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.760374 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ed6e515-cb3a-4258-86cb-a799985b3e4f-proc\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.760711 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.760475 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ed6e515-cb3a-4258-86cb-a799985b3e4f-sys\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.760711 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.760484 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ed6e515-cb3a-4258-86cb-a799985b3e4f-podres\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.760711 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.760480 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ed6e515-cb3a-4258-86cb-a799985b3e4f-lib-modules\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.767935 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.767901 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pjpz\" (UniqueName: \"kubernetes.io/projected/5ed6e515-cb3a-4258-86cb-a799985b3e4f-kube-api-access-8pjpz\") pod \"perf-node-gather-daemonset-xm442\" (UID: \"5ed6e515-cb3a-4258-86cb-a799985b3e4f\") " pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:06.840834 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.840754 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5556bd56b8-hfvh8_8413a709-bee7-4265-ac32-9f867297c28e/console/0.log" Sep 06 11:04:06.867460 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.867429 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pw9j6\"/\"default-dockercfg-xq2z4\"" Sep 06 11:04:06.875414 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:06.875384 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:07.146254 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:07.146225 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442"] Sep 06 11:04:07.147802 ip-10-0-139-139 kubenswrapper[2574]: W0906 11:04:07.147773 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5ed6e515_cb3a_4258_86cb_a799985b3e4f.slice/crio-9b4099c4636ea241c297fd44542778c601e61ce5e1a4911a83052acbf231dced WatchSource:0}: Error finding container 9b4099c4636ea241c297fd44542778c601e61ce5e1a4911a83052acbf231dced: Status 404 returned error can't find the container with id 9b4099c4636ea241c297fd44542778c601e61ce5e1a4911a83052acbf231dced Sep 06 11:04:07.368763 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:07.368733 2574 scope.go:117] "RemoveContainer" containerID="cfe1a23505eb65f9f9031166f6e0b5c48c221974d1ca710eb772a16b53a5834d" Sep 06 11:04:07.369258 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:07.369219 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" event={"ID":"5ed6e515-cb3a-4258-86cb-a799985b3e4f","Type":"ContainerStarted","Data":"7c3e9d520d02455d016ba12f98c62946480b64a867f4a2c5fa6d4e28c0372eb4"} Sep 06 11:04:07.369387 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:07.369287 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:07.369387 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:07.369314 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" event={"ID":"5ed6e515-cb3a-4258-86cb-a799985b3e4f","Type":"ContainerStarted","Data":"9b4099c4636ea241c297fd44542778c601e61ce5e1a4911a83052acbf231dced"} Sep 06 11:04:07.383888 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:07.383836 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" podStartSLOduration=1.383816237 podStartE2EDuration="1.383816237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 11:04:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 11:04:07.382012201 +0000 UTC m=+2040.964501831" watchObservedRunningTime="2026-09-06 11:04:07.383816237 +0000 UTC m=+2040.966305871" Sep 06 11:04:08.687779 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:08.687750 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bdp7c_a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1/dns/0.log" Sep 06 11:04:08.706456 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:08.706428 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bdp7c_a3a3fa7c-6e6f-4572-b490-c658b8ff6bc1/kube-rbac-proxy/0.log" Sep 06 11:04:08.838875 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:08.838845 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hn5z4_2da0a8a3-afbc-41fc-bd56-69500eb10934/dns-node-resolver/0.log" Sep 06 11:04:09.411406 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:09.411371 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-66799f8b9-wfgqs_ce928fb9-c6bb-4cfd-8e5c-e803f9990fb5/registry/0.log" Sep 06 11:04:09.432542 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:09.432517 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4h9s4_41f8d38b-5235-4d1e-9840-2931fc5cd2b6/node-ca/0.log" Sep 06 11:04:10.387190 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:10.387158 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk_2ce36125-a847-43b9-bab2-29bd4bf0f8f5/istio-proxy/0.log" Sep 06 11:04:10.913073 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:10.913038 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-m7ftd_0719fc1d-2dea-450e-b161-8e2361629cf0/discovery/0.log" Sep 06 11:04:11.083007 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:11.082960 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jdfbt_7d9b8533-457c-43a4-a96e-c42b3ad07119/payload-processing/0.log" Sep 06 11:04:12.000755 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:12.000722 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rh5rj_bf8a2f30-8ef2-4cb9-9d32-4beac035a680/serve-healthcheck-canary/0.log" Sep 06 11:04:12.488444 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:12.488410 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-56h9s_9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df/kube-rbac-proxy/0.log" Sep 06 11:04:12.510792 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:12.510755 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-56h9s_9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df/exporter/0.log" Sep 06 11:04:12.529475 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:12.529443 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-56h9s_9e27c91d-308c-40fc-a3a0-5fd8c9c4a8df/extractor/0.log" Sep 06 11:04:13.384227 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:13.384198 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pw9j6/perf-node-gather-daemonset-xm442" Sep 06 11:04:15.123679 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:15.123656 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zn5wj_ea239aef-7690-41a3-8a78-5e216efc0b45/server/0.log" Sep 06 11:04:15.250620 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:15.250591 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tvq92_7d28b3ec-4bec-4f03-a4d2-c724aedaf2b2/manager/0.log" Sep 06 11:04:15.363278 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:15.363251 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645f664568-2zfbs_1d45ffaa-1462-4ec1-a397-9cca1f483cce/manager/0.log" Sep 06 11:04:15.378724 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:15.378651 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-645f664568-2zfbs_1d45ffaa-1462-4ec1-a397-9cca1f483cce/copy-manifests/0.log" Sep 06 11:04:16.942167 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:16.942134 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59b7bf7857-tmvq6_8b1c0066-771d-4f54-9d63-60718e979073/manager/0.log" Sep 06 11:04:16.965987 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:16.965959 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-8w2qk_00760b1e-0642-4080-b4a4-5b0484257ec4/openshift-lws-operator/0.log" Sep 06 11:04:17.210070 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:17.209972 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk_2ce36125-a847-43b9-bab2-29bd4bf0f8f5/istio-proxy/0.log" Sep 06 11:04:17.748273 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:17.748237 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-m7ftd_0719fc1d-2dea-450e-b161-8e2361629cf0/discovery/0.log" Sep 06 11:04:17.879156 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:17.879106 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jdfbt_7d9b8533-457c-43a4-a96e-c42b3ad07119/payload-processing/0.log" Sep 06 11:04:19.379192 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:19.379157 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-llhlk_2ce36125-a847-43b9-bab2-29bd4bf0f8f5/istio-proxy/0.log" Sep 06 11:04:19.909786 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:19.909749 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-m7ftd_0719fc1d-2dea-450e-b161-8e2361629cf0/discovery/0.log" Sep 06 11:04:20.057402 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:20.057363 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jdfbt_7d9b8533-457c-43a4-a96e-c42b3ad07119/payload-processing/0.log" Sep 06 11:04:22.617950 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:22.617897 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 11:04:22.719978 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:22.719944 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ggl9h_d3f60912-3413-4b9f-acb3-8d38dd276a82/konnectivity-agent/0.log" limit=1024 Sep 06 11:04:23.235957 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:23.235926 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6rcdb_a28d14f7-334e-4b14-b75b-14619d4c4f4f/kube-multus/0.log" Sep 06 11:04:23.296449 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:23.296421 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hfgzs_52513a46-7d58-48a5-a6a9-7633b34482c7/kube-multus-additional-cni-plugins/0.log" Sep 06 11:04:23.315556 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:23.315530 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hfgzs_52513a46-7d58-48a5-a6a9-7633b34482c7/egress-router-binary-copy/0.log" Sep 06 11:04:23.333956 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:23.333925 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hfgzs_52513a46-7d58-48a5-a6a9-7633b34482c7/cni-plugins/0.log" Sep 06 11:04:23.352682 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:23.352656 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hfgzs_52513a46-7d58-48a5-a6a9-7633b34482c7/bond-cni-plugin/0.log" Sep 06 11:04:23.371266 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:23.371239 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hfgzs_52513a46-7d58-48a5-a6a9-7633b34482c7/routeoverride-cni/0.log" Sep 06 11:04:23.390328 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:23.390304 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hfgzs_52513a46-7d58-48a5-a6a9-7633b34482c7/whereabouts-cni-bincopy/0.log" Sep 06 11:04:23.409324 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:23.409301 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hfgzs_52513a46-7d58-48a5-a6a9-7633b34482c7/whereabouts-cni/0.log" Sep 06 11:04:23.846767 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:23.846734 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tg7pf_23d754da-9be5-4fe4-9d31-0ce5f1c3b63e/network-metrics-daemon/0.log" Sep 06 11:04:23.860562 ip-10-0-139-139 kubenswrapper[2574]: I0906 11:04:23.860512 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tg7pf_23d754da-9be5-4fe4-9d31-0ce5f1c3b63e/kube-rbac-proxy/0.log"